1. Performance Base Logistics for the Ground / Air Task Oriented Radar, 5 February 2004 Presented by John Sells, Product Support Manager - John.sells@tobyhanna.army.mil DSN 795-7585 Comm. 570-895-7485

2. Discussion Points DOD Acquisition Management Framework The beginnings of PBL IPT Roles & Responsibilities Business Case Analysis PBA Notional Approach PBA development Recommendations Questions & Ideas

3. New DoD Acquisition Management Framework – Five Phase Overview Pre-Systems Acquisition Systems Acquisition Sustainment Concept Technology Systems Development Production & Operations & Development Demonstration Deployment Support Concept Component System System Low-Rate Full-Rate Sustain Disposal Exploration Advanced Integration Demonstration Initial Production Demonstration Production ABC IOC FOC Performance Based Logistics Principles Decision to Enter Concept Refinement Phase FRP Decision Review Design Readiness Review ICD CDD CPD Funding Early and Continuous Testing RDT&E RDT&E SDD Procurement Operations & Support BA3 BA4 BA5 APN WPN MILCON OPN O&M,N Initial Capabilities Document Capabilities Development Document Capabilities Production Validated Requirements … Minimum KPPs Set DT&E OT&E FOT&E Combined DT&E and OT&E Evolutionary Acquisition End-State Requirement Incremental Acquisition Spiral Development Desired Capability Known End -State Not Known Technology Maturation & User Feedback Drive Future increments Block Upgrades Product Support Planning Develop Product Support Concept Identify Product Support Integrator Update periodically Design the Support Develop Support for the Design Establish PBA Document Program Initiation & Decision to Enter System Development & Demo Phase Concept Exploration Technology Development Decision to Enter Technology Development Phase Program Initiation & Decision to Enter System Development & Demo Phase Support Strategy Concepts Acquisition Strategy Readiness & Total Ownership Cost Objectives R&D System Design for Operational Effectiveness Goal #1: Design for Supportability Goal #2: Design & Develop Support System Produce Support System Goal #3: Acquire & Deploy Supportable System Demonstrate Supportability & Life Cycle Affordability Demonstrate Supportability & Life Cycle Affordability Support Concept Inputs: ICD AOA TEMP User Needs S&T Activity ATD ACTD Joint Warfighting Experiments Inputs: ICD AOA Plan CONOPS Inputs: Acquisition - Strategy - Program Baseline CDD AOA TEMP SDD RFP Develop KPPs for CDD Develop Key Logistics Parameters (KLPs) Develop Support Metrics Establish PBA

4. New DoD Acquisition Management Framework – Documentation & Decisions Pre-Systems Acquisition Systems Acquisition Sustainment Concept Technology Systems Development Production & Operations & Development Demonstration Deployment Support Concept Component System System Low-Rate Full-Rate Sustain Disposal Exploration Advanced Integration Demonstration Initial Production Demonstration Production ABC IOC FOC Decision to Enter Concept Refinement Phase FRP Decision Review Design Readiness Review Program Initiation & Decision to Enter System Development & Demo Phase Concept Exploration Technology Development Decision to Enter Technology Development Phase Program Initiation & Decision to Enter System Development & Demo Phase Concept Refinement: - Validated ICD - Approved AOA Plan Completed in Phase: - Conduct AOA, Refine Concept - Develop Tech Development Strategy Milestone A: - Initial Capabilities Document (ICD) validated & approved - Milestone Decision Authority (MDA): Approves Tech Development Phase Tech Development Strategy (TDS) Exit Criteria for SDD Phase Acquisition Decision Memo (ADM)  Completed in Phase: - Development of System Architecture - Demo of Technologies - Risk Reduction on Subsystems - Form Decision Making IPTs - Capabilities Development Document (CDD) Prepared by IPT - Conduct Cost Performance Trade-offs - Develop Acquisition Strategy (ASR) - Set Acquisition Program Baseline (APB) - Complete TEMP - Prepare C 4 ISR (if required) - Prepare Life-Cycle Cost Estimates - Prepare RFP for SDD Milestone B: - Entrance Criteria Validated CDD Validated requirements … Minimum Key Performance Parameters (KPP) set Mature Technology APB Affordability Determination Full Funding in FYDP - Formally initiates Acquisition Program MDA approves: - Entry into SDD Phase - Acquisition Program Initiation - ASR & APB - Exit Criteria for Production Phase - Acquisition Decision Memo (ADM) Completed in Phase: - Architecture validated thru integration demo of sub-systems - Design Readiness Review - Testing and Demonstration - Prepare RFP for LRIP - Capabilities Production Document (CPD) Milestone C: - Entrance Criteria Technology Maturity Approved CPD Fully Funded Acceptable:  Performance in DT&E  Manufacturing Risks  Interoperability  Operational Supportability  Affordability - Authorize entry into LRIP MDA Approves: - CPD - ADM - Acquisition Strategy & APB - LRIP & Later FRP Completed in Phase: - Testing ( DT, IOT&E) - Support & Deployment Plans - Prepare RFP for FRP - Prepare for FRP Decision All support elements in place to sustain system at acceptable performance levels identified in CPD and Performance Based Agreement (PBA) with war fighter Evolutionary acquisition of product support elements continues to sustain systems upgraded through spiral development of systems in reaching Full Operational Capability (FOC) over life cycle

5. New DoD Acquisition Management Framework – TLCSM & PBL Template Pre-Systems Acquisition Systems Acquisition Concept Technology Systems Development Development Demonstration Production &Deployment ABC Concept Exploration Technology Development Consider Key Logistics Criteria: Forecast Operational Environment Assess: - Functional Characteristics - Impact of System on Maintenance Capabilities Planned - Preliminary Manpower/Pers Requirements (Quantity, Skills, Uniform or Contractor) Compilation of Info and Requirements Initiate Development of Operating & Support Reliability Objectives, Benefits & Resource Requirements Assess Concepts & Technology in using: - Embedded Diagnostics - Prognostics - Other Maintenance Enablers Compile & Assess Data on Projected: - Sustainment Demand - Standardization - Required Support Equipment (SE) Develop ROM Life Cycle Costs (LCC) Key Logistics Info Compiled:  AOA Performance-Based Support Options  Identify Support Parameters for CDD  Market Analysis  Identify Risks/Risk Mitigation Planning Key Logistics Activities Completed:  Sustainment Parameters for CDD  Product Support Strategy for ASR  Logistics Metrics Criteria/Funding for APB  Logistics Considerations for TEMP Consider Key Logistics Criteria: More Discrete Requirements to Support Required Mission Operational Capabilities - Metrics to Drive Performance Outcomes (Reliability, Maintainability, & Availability - RMA) Manpower/Pers Requirements Refinement of LCC Support-Related Performance/Acceptance Criteria to be Demo during Testing Collection, Analysis & Eval of System RMA Performance Data to Determine Need for Design Changes (SDOE) Logistics Support Considerations Verify Support-Related Design Characteristics Identify Product Support Integrator (PSI) ISOR and Workload Assessment Develop Initial PBL BCA Set PSI Performance Outcomes/Requirements Develop PBL Product Support Concept w/PBA Key Logistics Info Compiled: Updated Support Strategy, Funding Required, Key Logistics Parameters (KLPs) PBL BCA Auditable ISOR Core Capability & Workload Assessment Key Logistics Activities Completed: Updated ASR, APB, CPD, & TEMP Acceptable Performance in DT&E Life Cycle Affordability Demonstrated Consider Key Logistics Criteria: Ability of Sustainment Strategy to Meet Mission Capability Requirements RMA Meets Performance Objectives Goals for Manpower/Pers Validated Final LCC Validated Product Support: - Completed BCA on PBL Approach Selected - Completed, Approved & Funded Product Support/Sustainment Approach to Include: › Documented PBA Between: - PM, PSI and Force Provider Defining System Operational Performance Requirements - PM, PSI and Support Providers which Include Required Support Metrics Necessary to Meet System Performance Requirements. Note: Support Providers may be Public, Private or Partnerships. Public providers can include depots, DLA, NAVICP, & DLA Distribution Depots - Comprehensive Review of Support-Related Performance & Acceptance Criteria for Pre-IOC Supportability Assessment: › Verify Implementation & Execution of PBA › Verify Funding of O&S to Required Levels Key Logistics Info Compiled: Updated Support Strategy within ASR Updated Logistics Parameters in APB Updated Logistics & Sustainment Criteria & Test Points PBA Signed (PM, PSI, Warfighter; PM, PSI and Providers) Key Logistics Activities Completed: Satisfaction of sustainment criteria in IOT&E PBAs Fully Funded Sustainment Program Pre-Initial IOC Review

6. Logistics Metrics Criteria Support Parameters Product Support Strategy (PSS) Logistics Considerations for TEMP Updated Support Strategy Assign PSI Finalize PSS Sign PBAs ISOR Core Assessment Supportability Demo Life Cycle Affordability Demo Establish PBAs Introduce Key Performance Parameters (KPP) From all of this, PBL Emerges…

7. COMMON RELEVANT OPERATING PICTURE The interaction at the grid points establishes the linkage of the Life Cycle Logistics elements. PBL is the Enabler for Force-centric Logistics Enterprise Public Private Partnerships Executive Agents Enterprise Integration Product Support Integrator Total Life Cycle Systems Management Condition-Based Maintenance + End-to-End Distribution Performance Based Logistics grid Force Provider

8. IPT Challenges

9. What should an IPT be able to do? Ability to have complete, accurate, real-time total asset visibility Ability to collect accurate, near real time data Ability to assess the data and make informed decisions Funds Flow Ability to fund effectively and manage execution Organize IPT’s around program requirements!

10. Focus on Total Life Cycle Systems Management ! DoD Logistics Value: MARSYSCOM Manages:: COMMON SUPPORT ACROSS WEAPONS SYSTEMS WEAPON SYSTEM-SPECIFIC SUPPORT ENTERPRISE SOLUTIONS EFFECTIVE METRICS 2-LEVEL MAINTENANCE TPS-59 CLAWS TPS-63TPS-73 MPQ-62 TPQ-46A G/ATOR Operational Availability END TO END CUSTOMER SUPPORT PUBLIC PRIVATE PARTNERSHIPS Organizational/Unit Measure: The IPT Challenge: Integrating ALL of the Solutions

11. Under Secretary of Defense for Acquisition, Technology and Logistics Assistant Secretary of Defense for Command, Control, Communications & Intelligence (C 3 I) RULES OF THE ROAD: A GUIDE FOR LEADING SUCCESSFUL INTEGRATED PRODUCT TEAMS,Revision 1,October 1999 DoD IPT Operational Structure

12. GATOR Overarching IPT (OIPT) GATOR Project Management IPT (PM IPT) Funding and Contracts IPT Integrated Test IPT Software Integration IPT Design and Engineering IPT Logistic IPT Production IPT Transition to Production IPT Fielding IPT COMM Sub- IPT Softwar e Sub-IPT Processors D isplays Sub-IPT OPFAC Sub- IPT Rqmts/Intero IPT G/ATOR Integrated Product Teams Structure Integrated Product and Process Development (IPPD) for the G/ATOR Program* * IPT structure from the SUPPORTABILITY PLAN FOR THE GROUND AIR TASK ORIENTED RADAR, Draft, 6 Jan 04 If this was a fight, Could the G/ATOR Program Management IPT defend themselves? Some of these IPT’s are requirements-driven (good), some are based on functions (not so good) IPPD is not easy – Managers MUST shift paradigms! Should be direct line from OIPT A lot of IPT requirements identified, but it may be a bit Chaotic…

13. Radar Systems (PM112) Battlespace and Air Defense Systems Product Group (PG11) Overarching IPT Working IPT’s: Supportability Cost & Performance Design & engineering Set goals for each IPT Focus on TLCSM Choose Integrated Solutions Benchmark & Build through experience Measure Performance for Continuous Improvement Be willing to change! MARCORSYSCOM Integrated Data Environment GATOR Integrating IPT Understand what you are currently doing. Integrated Product and Process Development Objectives * ILS Elements from the SUPPORTABILITY PLAN FOR THE GROUND AIR TASK ORIENTED RADAR, Draft, 6 Jan 04 Performance-Based Business Environment Evolving G/ATOR ILS Elements* Maintenance Planning Manpower and Personnel Supply Support Support Equipment Technical Data Training and Training Support PHS&T Naval Integration and Transportability Facilities NOTIONAL CONCEPT ON BUILDING IPT’S

14. The Business Case Analysis (BCA) The BCA links performance levels and resource requirements which enables a much more intelligent budgeting process. The BCA is a toolset for identifying Total Ownership Costs The BCA has the goal of identifying the BEST VALUE It is the BCA that helps determine the best choice for Product Support Integrators, and the parameters of the Performance Based Agreements! Bottomline – Senior leaders need all appropriate information to make good decisions. They will not stake their reputations simply on a good “story”.

15. What is a BCA ? Until recently, BCA was not a common term in DoD Formerly known by the terms: “economic analysis; cost/benefit analysis; cost of ownership analysis; analysis of alternatives; feasibility study; staff study” “The Business Case Guide”; by Dr. Marty J. Schmidt, Solution Matrix, Ltd; 2002 www.solutionmatrix.com “BCA is an extended form of cost/benefit analysis where the alternatives’ total costs for satisfying a business need are weighed against the alternatives’ total benefits to determine an optimum solution. BCA goes one step further than cost/benefit analysis and links each alternative to fulfilling the strategic objectives of the organization, compliance with performance measures and impact on stakeholders. BCA identifies which alternative allows an organization to optimize mission performance given cost and other constraints. It not only includes analysis of cost, but also addresses qualitative and subjective factors directly impacting the decision-making process, along with risk and sensitivity analysis. The output of a BCA is an analysis of all viable alternative business strategies, along with recommendations for proceeding with the best value alternative.” “A Business Case is a decision support tool that projects the likely financial results and other business consequences of an action. It shows the cash flow consequences, over time, and – most important – includes the methods and rationale used for quantifying benefits and costs.” Dr. Jacques Gansler,

16. Why Go Through with a BCA ? BCA’s required by Statute for all proposed “Prime Vendor” support arrangements; and by USD/ATL Section 346, 1999 NDAA, as amended by Section 336, 2000 NDA 13 Feb 2002 USD/ATL Memo to Services re: PBL Implementation Schedule There is no current standard OSD format (in work) BCA’s have several uses in implimenting PBL Assesing viability of overall strategy, workload allocation, and cost comparisons BCA’s are a toolset of PBL, useful in many circumstances.

17. BCA Structure A.Introduction Defines what the case is about (the subject) and why (its purpose). Presents the objectives addressed by the subject of the case. B.Methods and Assumptions Design elements analysis methods and rationale that fix the boundaries of the case (whose costs and whose benefits examined over what time period). Outlines the rules for deciding what belongs in the case and what does not, along with the important assumptions. C. Business Impacts The main business case results – financial and non-financial business impacts expected in one or more scenarios. D. Sensitivity and Risks Shows how results depend on important assumptions (“what if”), as well as the likelihood for other results to surface E. Conclusions and Recommendations Recommends specific actions based on business objectives and the results of the analysis. From: “The Business Case Guide”; by Dr. Marty J. Schmidt, Solution Matrix, Ltd; 2002

18. Sample BCA Format Executive Summary Introduction/Overview Subject, Purpose and Objectives Background Organization Major Assumptions and Constraints Scope of Analysis Financial Metrics Used and Defined Analysis Methodology The Cost and Benefit Model Used (Explanation of Cost Types and Categories) Description of Alternatives Costs and Benefits Over Time Financial Analysis Non-quantitative Factors, Criteria, and Rationale for Their Use Comparison of Alternatives (quantitative and qualitative) Sensitivity Analysis Risk Analysis Conclusions and Recommendations Results Rationale Choice of Scenarios for Implementation Strategy and Tactics to Optimize Results Introduction Methods and Assumptions Business Impacts Sensitivity and Risks Conclusions & Recommendations

19. Developing a Performance Based Agreement (PBA)

20. Notional/ Sustainment Field Strategic Product Support Integrator (PSI) Force Provider Program Manager (PM) Performance-Based Agreements (PBA) Notional Approach  Agree to provide the funds for products and services rendered  Agrees to perform all field level operations in accordance with established Field Manuals and  Technical Manuals  Agrees to perform all field level maintenance in accordance with established Technical Manuals, Lube Orders, etc.  Agrees to using trained operators/maintainers, and/or that operators/maintainers receive proper training  Agrees to perform all supply chain operations in accordance with established policy  Agrees to perform other logistics transactions in accordance with established policy  Agree to provide the negotiated Operational Availability, Mission Capability, Operational Readiness  Agree to provide the Force Provider a voice on the Supportability IPT (SIPT)  Agree to provide funding to the PSI commensurate with the defined requirements  Agree to provide the PSI a voice on the Supportability IPT (SIPT)  Agrees to perform the duties of the PSI as identified in the PBA  Agree to serve on the SIPT  Agree to comply with Contractors on the Battlefield policy  Agree to maintain a transparent supply chain with the field level  Agree to provide access to all performance data  Agree to integrate the supply chain activities of all Product Support Providers (PSPs)  Agree to search for increases in reliability

21. BARRIERS AND METRICS Contract Types: BCA: Metrics:  Firm Fixed Price  Cost Plus Incentive Fee  Economic Analysis (EA)  Best Value Analysis (BVA)  Business Case Analysis (BCA) Barriers:  Funding Flow  Stability of Funding  Funding for Long Term Contracts  PBL Becomes “Must Pay” Bill  Business Risks  Working Capital Fund (WCF)  Guidance Not Evolving Rapidly Enough  Incentivizing/Disincentivizing Organic Sources  Statutory Rules  Contractor Limitation on Unit Level Performance  Contractors on the Battlefield Policy  Lack of PBL Training  Lack of or cost of Data Collection System  Non-Mission Capable Supply (NMCS)  Mission Capable Deliveries  Non-Mission Capable Maintenance (NMCM)  Supply Availability  10% Reduction in O&S Costs  Customer Wait Time (CWT)  Mean Time To Repair (MTTR) Days  Fill Rate  Operational Availability (Ao)  Operational Readiness Rate (ORR)  In-Flight Abort Rates  Cost of Flight Hour  Turn-Around-Time (TAT)  Logistics Maintenance Ratio (LMR)  Training Equipment Availability Rate (TEAR)  Built In Test (BIT) Accuracy  Cost Per Mile to Operate  Operational Effectiveness  Contractor Performance Factor (CPF)  Non-Chargeable Down Time (NCDT)

22. Developing a PBA Overview Purpose Document, in terms of performance, the relevant support requirements Document what that specified level of performance will cost. Background Brief description of program and decision criteria in choosing this solution. Metrics Used to measure how well a program is meeting the objectives Identification of realistic, quantifiable, and measurable metrics May change as requirements of the program evolve Methodology: Use of warfighter performance requirements to influence the design Identification of all stakeholders roles and responsibilities (IPT’s), for the collection, processing, analysis and reporting of performance data Identification of the source and data to be collected Formal performance review & dispute resolution process Fightin’ words: “Where does PBL fit in with how we are doing business today?

23. Developing a PBA Overview (continued) Roles and Responsibilities Includes Program Office, warfighter, Product Support Integrator Lead, Product Support Integrator Team. Schedule Major milestones and criteria for demonstrating successful accomplishments Period of Performance Not a one-time event. Formally establishes an ongoing relationship between Government/Industry and the Warfighter throughout the weapon system life cycle. Will require continual management throughout the weapon system life cycle System evolution may lead to requirements change Approving Officials Should include: Program Executive Office Warfighter/User Product Support Lead Integrator Program Manager

24. Developing a PBA Template Performance Based Agreement Sections Objective and Scope Content Roles & Responsibilities Performance Measures Revisions and Flexibility Accountability and Oversight Existing Agreements Contingency Agreements Execution of Agreement

25. Performance Based Agreements Preface Provides a common framework, a ‘checklist’ to consider, that may involve one or more organizations Guidance, not direction…as the PBA matures, “best practices” will evolve and define the necessary PBA agreement restructuring. Consider the following elements when developing a Performance Based Agreement: Objective and Scope Content Roles and Responsibilities Performance Measures Revisions and Flexibility Accountability and Oversight Contingency Agreements Execution of Agreement

26. PBA Objective and Scope Objective: Establish realistic, quantifiable, and measurable metrics between MARCORSYSCOM and Product Support Providers Scope: PBA will address a basic set of criteria: Document customer requirements and establish a set of mutual expectations. Shall document the intent of the parties to review specific period of performance for the purpose of measuring the effectiveness of the product support providers. Ideally expressed in contract-like terms which outline the desired outcome. Incentives and Penalities

27. PBA Intent MARCORSYSCOM and the customer sign the agreement to participate in the collaborative effort to design, build, field, and support G/ATOR Clear identification of the customer performance and support requirements that will be applied The program requirements will be quantified as metrics

28. PBA Roles & Responsibilities Should be clearly delineated in each individual PBA so that each stakeholder clearly understands the performance level to which they’ve subscribed. Key aspects include: Signature authority and the ultimate responsibility for monitoring, managing and executing all portions of the PBA agreement lies with the individuals designated therein. Responsibility for executing the portions of the agreement specific to the PBA and setting the detailed arrangements for accomplishing the execution of each portion of the agreement, documenting performance targets, and monitoring through measurements lies with the signatories to the agreement.

29. PBA Performance Measures Should be carefully evaluated and selected to measure the specific outcome that have meaningful impact on acquisition strategy. Potential performance measures: Operational Availability (Ao) Operational Readiness Rate (ORR) Turn-Around-Time (TAT) Logistics Maintenance Ratio (LMR) Built In Test (BIT) Accuracy Should establish timelines for the performance reviews. identification of metrics data collectio/validation base lining processes Should include procedures for reconciliation of differences Bring in your legal staff at this point! Feedback mechanism(s) should include Notification and Resolution procedures as well as Escalation Procedures.

30. PBA Revisions and Flexibility Unanticipated changes may necessitate a renegotiation Flexibility enablers: Envision and document a range for each metric to accommodate for fluctuating resources PBA’s are long term(7-10 years):review periodically! Synchronize the PBA review with the planning and budgeting process (POM) Subsequent reviews should not require a “clean sheet negotiation” effort Defining the resource level required to support specific performance thresholds is a significant step towards reducing Total Ownership Cost

31. PBA Accountability and Oversight PBAs provide for equitable and consistent expectations and enforcement for all parties Guidelines: Do you have adequate resources (i.e. funding, manpower support) necessary to achieve the required support levels? Are incentives for performance by organic providers considered and clearly defined within scope of PBA? Contingency Agreements PBAs should include provisions for modification for agreed-upon periods of time under contingency situations This including clauses that would be activated as a result of mission- related contingencies.

32. PBA Summary Performance Based Agreements are a critical element in implementing PBL Define Expectations of Force Provider Define range of support requirements Basis for negotiating support contracts Ensure accountability in meeting Warfighter requirements Getting them right is critical!

33. RECOMMENDATIONS Organize IPT’s around things that are core to your business, not around functions. Set goals for each IPT. Benchmark, stay current, be willing to change, keep improving Execute with excellence. Understand what you are currently doing Focus on Total Life Cycle Systems Management. Choose integrated solutions. Measure performance for continuous improvement. Get the right information to the right people. Build through experience.

34. Acknowledgments & thanks! Randy Fowler, Director, Center for Logistics and Sustainment Curriculum Development, Defense Acquisition University Kathie McGuire, Headquarters DLA Action Officer for USMC/Army Weapon Systems Resource Consultants, Inc. 2650 Park Tower Drive, Suite 800 Vienna, VA 22180 John L. Tyler, Lead Logistician, MARCORSYSCOM, BMADS, PM-Radar Systems George Perise, Log.Sec Corporation 4150 Weeks Drive, Warrenton, VA 20187