Cost Research Symposium 4/15/2018 AT&L Perspective on Joint Capabilities & Defense Cost Research Opportunities Good morning. Thank you, Rick and Russ, for inviting me to speak to your Cost Research Symposium. It’s great to be with you today and I want to share with you an AT&L perspective on our #1 job—providing capability to the warfighter at an affordable cost. I want to address AT&L’s perspective on doing that job, the value of good cost estimates, and the risk and metrics we need. I want to consider all of that across the Big A, Acquisition, which includes requirements, little a, acquisition, and logistics. As most of you know, I’m the Director of Acquisition Resources & Analysis. And I am the executive secretary to the Defense Acquisition Board, the AT&L point person on PPBES, and I’m responsible for Congressional issues. I also deal with other issues like responses to GAO and IG audits, and aspects of the Business Management Modernization Program, or BMMP, which I’ll talk a little about in a few minutes. I work directly for the Under Secretary of Defense for Acquisition, Technology and Logistics. Mr. Michael Wynne is in that position right now. However, the current nominee for this position, is Mr. Ken Krieg, Director, Program Analysis & Evaluation. He is nominated to be the next permanent USD(AT&L). Dr. Nancy L. Spruill Director, Acquisition Resources & Analysis Office of the Under Secretary of Defense (Acquisition, Technology, & Logistics) May 26, 2005

Cost Research Symposium 4/15/2018 Agenda Transformation Challenge AT&L Focus Spectrum of Acquisition Transformational Initiatives Cost Research Challenges Today I’ll focus on several challenges I think we face in meeting the materiel needs of the warfighter. I’ll discuss the transformation challenge from an acquisition perspective and I’ll address some of the Department’s major transformation initiatives. I’ll give several examples of some initiatives going on in my office to support transformation. And I’ll wrap up with some cost research challenges. I believe transformation is more than a buzzword in the Pentagon. I believe it is what has spurred us, in the past few years, to make a remarkable number of changes in how we fight this nation’s wars today and that it is what will allow us to do so, even faster and better, in the future. In order to change how we fight, we must change our business practices. And we must transform our acquisition process into one that is adaptive and, yet, can also deal with routine, or deliberate, needs. It must also be accountable.

Cost Research Symposium 4/15/2018 “The United States will … transform America’s national security institutions to meet the challenges and opportunities of the twenty-first century.” President George W. Bush September 2002 “The Department currently is pursuing transformational business and planning practices such as adaptive planning, a more entrepreneurial, future-oriented capabilities- based resource allocation process, accelerated acquisition cycles built on spiral development, out-put based management, and a reformed analytic support agenda.” Secretary of Defense Donald Rumsfeld Transformation Planning Guidance April 2003 So why is DoD transforming? As you know, we need to remake ourselves both for the challenges we face in fighting a war on terrorism and to get the most warfighting out of our appropriated $s—which after all are our hard earned tax dollars. And, transformation is a journey - there will be no moment at which the Department is “transformed”. The Secretary is building a culture of continual transformation.

Cost Research Symposium 4/15/2018 “DoD has a long way to go to ensure that our acquisition (and sustainment) process achieves the appropriate jointness and interoperability needed in the 21st Century…As we move forward with the QDR, we absolutely must transform the acquisition (and sustainment) process.” SecDef Snowflake October 21, 2004 The challenge to the Acquisition Community is clear. As Secretary Rumsfeld says in his October “Snowflake” our acquisition process has a long way to go to meet the challenges of the future. We must transform in order to provide the best fighting force for the nation. Now you all know what a snowflake is, don’t you? It isn’t what falls on the beltway and snarls traffic. It is the Secretary’s way of getting our attention. And there are days when there are so many snowflakes that we’d prefer the real ones. So the Secretary wants us to transform. And not just the acquisition process that supports the development of long lead items, but more importantly the acquisition process that supports the things that the warfighter, on point, needs today. AT&L is rising to this challenge – to meet both the adaptive and more routine, or deliberative, needs of the warfighter. Need an adaptive and deliberate acquisition processes

Cost Research Symposium 4/15/2018 The President’s Management Agenda (PMA) “…dedicated to ensuring that the resources entrusted to the federal government are well managed and wisely used. We owe that to the American people.” SECDEF - Transformation and Joint Warfighting DoD Transformational Planning Guidance FY 2005 Top 10 Priorities Balanced Score Card / Risk Management Framework So how are we rising to the Secretary’s challenge? We considered the goals, objectives and initiatives of our senior leadership and aligned our AT&L goals to the President’s Management Agenda and Secretary Rumsfeld’s guidance and initiatives. AT&L’s seven goals, shown here, are specifically targeted to drive performance, directly contributing to our joint warfighting strategy and the transformation of our DoD business processes. I want to talk about three of those goals. Acquisition Excellence with Integrity Logistics: Integrated and Efficient Systems Integration & Engineering for Mission Success Technology Dominance Resources Rationalized Industrial Base Strengthened Motivated, Agile Workforce AT&L Seven Goals Support PMA and SECDEF

Acquisition Excellence with Integrity Cost Research Symposium 4/15/2018 Acquisition Excellence with Integrity Bring Joint Capabilities Perspective to Acquisition Conduct Senior Leadership Reviews for each Functional Capability Area Re-structure PDM to Enforce Results of Senior Leadership Reviews in the Resource Process Transition from “System Focused” to Capabilities-Based DAES Reviews Increase Accuracy and Credibility of Cost Estimates Shorten Acquisition Cycle Time Increase use of evolutionary acquisition including spiral development Maximize use of mature and commercial technology Expand use of technology demonstrations (ACTD/ATD) - enter acquisition process “closer to production”. Battlespace Awareness Command & Control AT&L’s #1 goal is Acquisition Excellence with Integrity. It is about improving the efficiency, effectiveness, and performance of our acquisition processes, while ensuring the highest standards of conduct. One of the keys to improving our acquisition processes is to look at capabilities and to see individual weapons systems through the capability lens. A consequence of this approach is that it ensures we look at joint warfighting capabilities.  We look at how a new system will contribute to the entire capability area and to the joint battle. We also think in broader terms of what it takes to field and sustain capabilities, not just individual weapon systems. So as a step in this direction, we’ve instituted Capability Area Reviews at the Defense Acquisition Board, or DAB, level. And we’ve reorganized the Defense Acquisition Executive Summary, or DAES process, our top-level review for major programs, around capabilities. That process now reviews weapons systems within their own capability area.

Acquisition Excellence with Integrity Cost Research Symposium 4/15/2018 Acquisition Excellence with Integrity Bring Joint Capabilities Perspective to Acquisition Conduct Senior Leadership Reviews for each Functional Capability Area Re-structure PDM to Enforce Results of Senior Leadership Reviews in the Resource Process Transition from “System Focused” to Capabilities-Based DAES Reviews Increase Accuracy and Credibility of Cost Estimates Shorten Acquisition Cycle Time Increase use of evolutionary acquisition including spiral development Maximize use of mature and commercial technology Expand use of technology demonstrations (ACTD/ATD) - enter acquisition process “closer to production”. Battlespace Awareness Command & Control Now let’s talk about integrity. Our cost estimates must reflect the true total ownership costs and be accurate enough to enhance our credibility with Congress and the American people.  We must ensure we can deliver at an affordable cost. And we must have credibility with our customers—the warfighter. Evolutionary acquisition is a key tool here. It pushes systems to the warfighter faster by fielding increments of ever-increasing capability. Spiral development will ensure systems maintain a technology edge through user feedback combined with new/improved technology. Finally, we are emphasizing another aspect of acquisition with integrity. In light of the ethical violations of Darleen Druyun and Mike Sears, USD(AT&L) has introduced a number of initiatives. Last October he directed the acquisition workforce take a new Ethics Training module. It was specifically designed to address proper standards of conduct between defense contractors and government employees. He also directed a Defense Science Board study to look for ways to improve integrity in the acquisition process, he directed DCMA’s Deputy Director to head a review of all acquisitions overseen by Darleen Druyun over the last 10 years. As a result this review identified 8 cases showing anomolies, which were referred to the IG for further review. Also USD(AT&L) arranged for all the contractor protests to be moved from the Department to GAO. He has provided a memo on integrity to the entire acquisition workforce and will continue to stress its importance in many ways.

Acquisition Excellence with Integrity Cost Research Symposium 4/15/2018 Acquisition Excellence with Integrity Bring Joint Capabilities Perspective to Acquisition Conduct Senior Leadership Reviews for each Functional Capability Area Re-structure PDM to Enforce Results of Senior Leadership Reviews in the Resource Process Transition from “System Focused” to Capabilities-Based DAES Reviews Increase Accuracy and Credibility of Cost Estimates Shorten Acquisition Cycle Time Increase use of evolutionary acquisition including spiral development Maximize use of mature and commercial technology Expand use of technology demonstrations (ACTD/ATD) - enter acquisition process “closer to production”. Battlespace Awareness Command & Control Now before we leave AT&L’s #1 goal, let me note that it specifically calls out cost estimating as a key element of the goal. I’ll talk, in a few minutes, about where I think the Cost Community should focus their Cost Research to support this goal. But the point I want to make here is that without good cost estimates, AT&L can’t achieve our #1 goal and without good cost estimates, the Department will be stuck with an acquisition process that isn’t under control. That is, we’ll always get less than we planned for and/or it will always cost most than we thought.

Logistics: Integrated and Efficient Cost Research Symposium 4/15/2018 Logistics: Integrated and Efficient RFID Enterprise Integration Expand End-to-End Policies Reliable Delivery Weapon System Support Sustain Weapon System Material Readiness Corrosion Control Logistics: Integrated and Efficient AT&L’s 2nd goal is Logistics: Integrated and Efficient. It sets the direction for logistics transformation within the Department. It directly impacts Joint Warfighting operations today and future capability. This goal consists of several initiatives shown here: The use of RFID to capture in transit visibility of resources as they make their way to the warfighter; Moving to enterprise integration business systems and processes; End-to-end management of logistics, which reduces logistics handoffs and ensures reliable delivery of products and services; Weapon system support strategies founded on performance based logistics, which drives toward higher availability for weapon systems; Designing OUT logistics requirements through high reliability systems and reducing the deployable logistics footprint of operational and support forces; and Corrosion control, which mitigates the impacts of corrosion on resources which otherwise would ultimately drive up cost. The USD(AT&L) is now the Defense Logistics Executive, just like he is the Defense Acquisition Executive. He has renewed emphasis on life cycle management throughout acquisition and sustainment.

Cost Research Symposium 4/15/2018 Resources Rationalized Resource joint warfighting capabilities (Link resources to capabilities-based approach to enhance Joint warfighting capability) Rationalize infrastructure to support joint warfighting capabilities (BRAC, RRPI, Global Posture/Basing Strategy) Achieve domain owner objectives (Fundamental business process change and streamlining) Rationalize contracting (Business process streamlining) Enhance outsourcing / competitive sourcing (Revised OMB A-76) Our Goal #5 is Resources Rationalized. The idea here is that in order to ensure a Joint Warfighting capability, we must rationalize our business processes and infrastructure. We must make optimum use of our resources to drive Joint Warfighting capability and to ensure that our highest priorities within the Department are resourced. Rationalization of our infrastructure and contracting will allow us to transform common business-oriented support functions, improve joint use of assets, and reduce the total cost of ownership. This is a global effort, which includes the Integrated Global Presence and Basing Strategy. That strategy addresses overseas assets and the BRAC process, which the Department, with AT&L lead, has just rolled out.

Cost Research Symposium 4/15/2018 Spectrum of Acquisition Adaptive Action Oriented Time Sensitive JRAC Service Rapid action Teams Rapid Acquisition Authorities Deliberate Process Oriented Cost Sensitive PPBS MDAP Resources Ok, so how do we transformation acquisition? How do we get inside the decision cycle of our enemies? How do we get body armor to the troops in less than 167 days? I don’t know. But I do know how to think about transforming acquisition and I hope the QDR lays out a plan for making it happen. Dee Lee has the lead on improving the acquisition process for the business practives/processes improvement integrated process team of the QDR. We need to start with what works—processes such as the ACTD process. And we need to look at possible hidden causes of acquisition problems. For example, could continued, even well intentioned, requirements growth be part of the problem? And we need a process that meets the near term requirements of the warfighter, while still focusing on the future. Our system is well suited to meet the needs of long lead items, but when it comes to rapidly providing the tools for the engaged warfighter that process is too slow. There is no consistent process across DoD by which we identify needs and provide a life-cycle solution while the soldier is still engaged. We need to have a clearly identifiable process which is adaptive to the situation and meets the close fight of the warfighter on point. This process also needs to be seamlessly integrated with the more deliberate process we all know and love which follows the multi-year PPBE process. Time

Cost Research Symposium 4/15/2018 Deliberate Process Here you see the current acquisition process that helps us turn requirements into acquisition solutions. Although there is some flexibility to this process, the arrow on the bottom is still the common theme—time, a good slug of it. This process is a good process for developing materiel solutions for our forces that are designed to meet future needs. It offers a methodical process that ensures that both the right solution is developed and that the appropriate level over oversight is present. What this process does not do is provide the flexibility that allows us to turn a warfighter need, identified TODAY in the foxhole, into a solution that is made available TODAY. Time

Cost Research Symposium 4/15/2018 DoD initiatives Army's Rapid Equipping Force (REF) Army uses the Rapid Fielding Initiative (RFI) Combating Terrorism Technology Task Force (CTTTF) Joint Capability Technology Demonstrations (JCTD) As many of you know, we have several initiatives ongoing throughout DoD that are designed to address the adaptive part of the spectrum of acquisition. Let me list a few. The Rapid Equipping Force, or REF. It is designed to rapidly identify and provide off the shelf solutions to an immediate warfighter need. This process does not, however, consider the entire life cycle for an acquisition solution. At some point, someone has to provide support for these systems. At that point, when a successful REF program hits the “standard” acquisition process is where we have problems. We don’t yet know very well how to make that transition. Another Army initiative is the Rapid Fielding Initiative (RFI). It equips soldiers in CONUS with all the necessary items they will need in the Area of Operations. These items are continually updated as the needs change. An OSD/AT&L-led initiative is the CTTTF. It is a process which quickly identifies emerging technologies in response to operator needs and provides funding for rapid prototyping, testing and evaluation. And there is the JCTD. It was developed to meet the concerns of the CoCOMs. It is a coherent approach that provides a means for joint S&T to transition without traditional barriers ( a “cradle to grave” model).

DoD initiatives (cont.) Cost Research Symposium 4/15/2018 DoD initiatives (cont.)   Laboratories develop technologies that meet immediate warfighter needs Joint IED Defeat Task Force The Joint Staff is implementing a new procedure for the rapid validation and resourcing of urgent operational needs Rapid Acquisition Authority Then there are the Defense Department Labs. They are exploited for potential technology solutions to meet the immediate needs of the operational commander. The CTTTF works closely with the labs to support its technology objectives. The Joint-IED Defeat TF focuses all counter IED efforts within the DoD while engaging outside sources of potential solutions to defeat current and future IED threats. Their mission is to Stay ahead of enemy innovation. The RAA is a congressional initiative. It allows us to streamline the rapid acquisition process by removing financial barriers. This statute allows the SECDEF to use up to $100M from any color of money to meet the immediate needs of the warfighter.  Although this is not new money, it removes the barriers from moving money across accounts to resource warfighter solutions.

Cost Research Symposium 4/15/2018 DoD initiatives Despite most necessary authorities (to push equipment/services to the warfighter) already existing, process very often still stifled Establish an OSD Joint Rapid Acquisition Cell (JRAC) Responsible to the Secretary of Defense through Under Secretary of Defense (Acquisition Technology & Logistics) Monitors, coordinates and facilitates meeting the Combatant Commander’s Urgent Operational Acquisition Needs Provides visibility of needs for special joint efforts such as the IED Task Force and the Combating Terrorism Technology Task Force “It's a lot easier if stuff is invented in IRAQ, then you can avoid a lot of bureaucracy and get it straight to the soldier.” Army Rapid Equipping Force The last initiative I want to mention is the JRAC. It was established by the Deputy Secretary to cut through bureaucratic impediments which slow our ability to meet urgent materiel and logistics solutions for the Combatant Commanders.  The JRAC focuses on: 1. What is best for the warfighter? 2. Near-term acquisition, materiel, and logistics solutions The capability needed must be quickly fieldable, supportable in place, affordable, and of acceptable risk. And the JRAC is headed by one of A&L’s best acquisition professional—who I gave up to do this job. And I’m proud of it.

Cost Research Symposium 4/15/2018 Transformational Initiatives “Transform business operations to achieve improved warfighter support……” Greater business speed, flexibility, agility to match warfighter needs Enable costs reductions/efficiencies in the business mission area “…..while enabling financial accountability across DoD.” Enhanced credibility via external measures/audiences Broad, accurate and timely information visibility for informed decision-making BMMP Realignment Principles DoD Business Enterprise Clarity Tiered Accountability Horizontal Business Transformation Focus Business Alignment to Warfighter Needs Capabilities, not Systems, as Deliverables Lets now move from the acquisition process to some transformational business initiatives the Department is focused on. The first is the Business Management Modernization Program or BMMP.  This program has been ongoing for the past several years and has gotten additional guidance from the FY2005 NDAA. It is now set up to creates strategic alignment between the Department’s mission, goals, and objectives and its business processes and systems. It places priority on delivering end-to-end business mission capability/functionality.

Cost Research Symposium 4/15/2018 Business Mission Alignment BMMP is organized around the five Core Business Missions in the department: Financial Management (FM); Human Resources Management (HRM); Real Property and Installations Lifecycle Management (RPILM); Weapon System Lifecycle Management (WSLM); and Material Supply and Service Management (MSSM). I have been designated to represent USD(AT&L) as the Weapon System Lifecycle Management Core Business Mission Area lead. And, J-8 and I will co-chair the Investment Review Board for this mission area. We will review programs and recommend to USD(AT&L) whether investment should be continued in support of the end-to-end Weapon System Lifecycle Management business process.

Defense Acquisition Management Information Retrieval (DAMIR) Cost Research Symposium 4/15/2018 Defense Acquisition Management Information Retrieval (DAMIR) A capability that…. Enables OSD to better perform its acquisition oversight role Is responsive to OMB and Congressional needs Leverages existing data sources Easily enables improvements to new/better data Reduces non-value added workload up and down the chain Promotes net-centric information principles Second is DAMIR. DAMIR is the first iteration of the Weapon System Lifecycle Management priorities and focuses on acquisition program visibility. It seeks timely access to accurate, authoritative, and reliable information supporting acquisition oversight, accountability and decision-making throughout the Department for effective and efficient delivery of Warfighter capabilities It enables OSD to better perform its oversight role, but at the same time reduces non-value workload up and down the chain. Currently, each program office and each service are using their own systems for program management and oversight. These systems are all independent. As a result, we are all working from different numbers and there is a tremendous amount of duplicate data entry. In the future, OSD will access the same data sources as the Program Offices, improving the timeliness and accuracy of the data and reducing the numerous number of phone calls to the Program Offices. Additionally, DAMIR will encompass the principles of net-centricity. For example, we will “pull” the data as needed from official sources vice having the Program Offices enter the same data into multiple systems.

Acquisition Oversight Vision Cost Research Symposium 4/15/2018 Acquisition Oversight Vision This is a depiction of how we envision the DAMIR capability will work. Leverage Service and OSD using net-centric tenets: enter data once, post before processing, access data where it resides, eliminate the need for duplicative data entry at all levels. OSD and Services will have access to the same information. Notice that the arrows are two way, meaning we will share our data with the Services as well as them sharing their data with us. SO HOW WILL IT WORK?

Cost Research Symposium 4/15/2018 DAMIR Structured Data (Purview) Web-based application that accesses Acquisition information from disparate sources. Consists of presentation layer, web services data “pull” software, and change management. Pull data from Services. Users share the same data.. Access provided to OSD, Joint Staff, Services, Program Offices and Congress. Unstructured Data Virtual Library. DAMIR is being developed and released in increments. The first increment – Purview -- is an Executive Information System, which was released last year. For the first time, it provided OSD analysts and decision makers, the Services and Program Offices with web based access to acquisition program and oversight information. The next increment of DAMIR is just being released. It includes a pilot that uses web services to pull earned value information from the Army’s oversight system. When the data pull is fully implemented, we will be able to retire our legacy data collection system, eliminate duplicate data entry, and thus ensure everyone is working from the same information. We will also be expanding our user community by providing Congress access to DAMIR, allowing them to view the same information we are using and eliminating the need to submit hard copy reports to the Congress. The increment just being released will also include the ability to access unstructured information through a virtual library.

Cost Research Symposium 4/15/2018 DAMIR Purview – FBCB2 Purview provides an easy-to-use, intuitive user interface. The user can customize the display, deciding which programmatic information he/she wants to see. This example is for the FBCB2 program. The Force XXI Battle Command Brigade and Below (FBCB2) program is a digital, battle command information system that provides integrated, on-the-move, timely, relevant battle command information to commanders. It provides them with continuous updates of a common picture of the battlefield. This is our “standard” quad chart view showing the mission and description of the program, the cost variance, the schedule, and the points of contact.

Cost Research Symposium 4/15/2018 DAMIR Purview – FBCB2 And this is one of our several Earned Value charts. It displays contract variance information – showing how well the contract is meeting its cost and schedule. Historically, we have had a 4 to 6 month lag before OSD received this information. We have now streamlined this process and are pulling EV data from the Army’s system, both reducing the time lag and ensuring we are all working from the same set of information.

Military Equipment Valuation (MEV) Cost Research Symposium 4/15/2018 Military Equipment Valuation (MEV) Vision Military equipment is properly accounted for and reported in time to be of use to decision makers and for financial reporting. Mission Identify and lead the development of short/mid-term and long term solutions to comply with Governmental Accounting Standards. Solutions include: policies, procedures, business enterprise architecture, requirements development and systems. Goals Determine the value of military equipment, such as aircraft, ships, combat vehicles, and weapons, and include this information in the periodic financial statements prepared by the DoD. Third, also contributes to BMMP. It is the Military Equipment Valuation process lead by my PP&E office. Military Equipment Valuation is a way to implement the 2003 federal accounting standard that military equipment (including modifications and upgrades) must be properly valued, capitalized, and depreciated. Military Equipment Valuation is important as it: Provides reliable and accurate information to decision makers – Total acquisition costs of assets will be consistently determined, decision makers will get comparable information over time and between programs, and it will allow better investment planning for replacements. MEV increases public confidence – OMB (Linda M. Springer,) says “We owe the taxpayers no less than a private company would to its investors in being able to account for their money.” And, it’s the law – and policy! (Chief Financial Officers Act of 1990, Federal Financial Management Improvement Act of 1996, OMB Circular A-127, President’s Management Agenda (2002), and Secretary of Defense Memorandum (July 19, 2001).

Military Equipment Valuation (MEV) Status Cost Research Symposium 4/15/2018 Military Equipment Valuation (MEV) Status Number of programs: 1,055 Valuations completed through February 2005: 547 Valuations in-progress: 147 FY2005 program valuations goal (cumulative): 968 Accounting systems interfaced with: 16 Property Accountability systems interfaced with: 28 Data elements in accounting/accountability systems: >600 Historically, DoD expensed the cost of military equipment at the time of acquisition; however, new accounting standards require the capitalization and depreciation of military equipment. We began implementing this standard in FY 2003 with a goal of completion by FY 2006. To date, we have identified 1,055 systems and completed valuations on 547 of these systems. We have interfaced with 16 accounting systems, 28 property accountability systems and over 600 data elements. We are on track to have completed 968 valuations by the end of this year and the remaining systems in FY2006.

Military Equipment Valuation – F/A–18 E/F Cost Research Symposium 4/15/2018 Military Equipment Valuation – F/A–18 E/F Program Balances Estimated Cumulative Cost of F/A-18 E/Fs in Service A $12,404.5 Accumulated Depreciation B $1,974.7 Net Book Value of F/A-18 E/Fs in Service C = A - B $10,429.8 Total Work-in-Process D $3,381.3 Cumulative Cost of F/A-18 E/Fs in Service and Work-in-Process E = A + D $15,785.7 Annual Depreciation Expense F $618.2 Estimated Cost per F/A-18 E/F Total Estimated Program Costs G $28,170.1 Total Number of F/A-18 E/Fs H 352 Estimated Cost per F/A-18 E/Fs I = G / H $80.0 Let’s take a look at the F/A-18 E/F as an example. To complete the evaluation, we had to value, capitalize and depreciate the F/A-18. The Program Balances section provides: the amount capitalized for F/A-18s (A), the value of F/A-18s not yet in service but in process of being produced (D), the amount of total program depreciation (B), the annual depreciation amount (F), and net book value (C). The Estimated Cost per F/A-18 E/F section shows how we determine the unit cost. So, for the F/A-18, we have a net book value of approximately $10 billion and a unit cost of $80 million. As of April 19, 2004 $ in Millions

Cost Research Symposium 4/15/2018 Cost Research Challenges Lets now move from the acquisition process to look at how cost research can support transformation, especially business transformation.

Cost Research Crossroads Cost Research Symposium 4/15/2018 Cost Research Crossroads A. Focus on pure cost estimating? OR B. Incorporate other real world factors that affect estimate results? A B Should you focus on pure cost research or incorporate other real world factors?

Cost Research Crossroads Cost Research Symposium 4/15/2018 Cost Research Crossroads Know Your Customer’s Needs and Estimate Drivers Identified Who is involved in the estimate? How is the estimate prepared? What is known (scope definition)? Other factors affecting the estimate. The answer is B. But you are smart cost researchers, so you knew that. Cost Research must continue to meet their mission. B is not theoretical work. You have to put your cost estimates in context. Is the program you are estimating have realistic schedules and affordable materials? Do you fully understand the high risk areas? Given other priorities, should you attempt to cost it? B. Incorporate other real world factors that affect estimate results.

Cost Research Opportunities Cost Research Symposium 4/15/2018 Cost Research Opportunities As of: 28 Apr 05 But what about thinking jointly and in terms of capability. Is that how you approach your work? Earlier I mentioned the Secretary recently tasked us to start planning along capabilities lines and to move to implement them throughout our processes. As you know, we are starting to have Capability Area Reviews. What kind of cost estimates should we be considering there? Surely the answer isn’t none. Here is the SecDef approved list, which contains 21 capability areas. They are the first “tier” of major operational and functional capabilities. We have to think about our programs as part of one or more of these—and everything about our programs in this context—what they contribute, when they are needed, how they are tested, costed, and supported. This is the first step toward something different from the Major Force Programs of the FYDP that have been in place for 40 years! SECDEF Memo approving 21 Joint Capability Areas signed May 6, 2005 First step toward something different from the Major Force Programs of the FYDP that have been in place for 40 years!

Cost Research Opportunities Cost Research Symposium 4/15/2018 Cost Research Opportunities As of: 28 Apr 05 Perhaps you can focus your team’s Cost Research resources toward solving a “Tier 2” capability issue. Notice the key word in all of theses capability areas—Joint. That makes it harder but that makes it more relevant to our planning and programming in today’s environment. SECDEF Memo approving Tier 1 & Tier 2 Joint Capability Areas signed May 6, 2005 First step toward something different from the Major Force Programs of the FYDP that have been in place for 40 years!

Cost Research Opportunities Cost Research Symposium 4/15/2018 Cost Research Opportunities As of: 28 Apr 05 And here are more Tier 2 potential study candidates. SECDEF Memo approving Tier 1 & Tier 2 Joint Capability Areas signed May 6, 2005 First step toward something different from the Major Force Programs of the FYDP that have been in place for 40 years!

Cost Research Symposium 4/15/2018 Opportunity Costs Funding future Cost Research estimates are: Basis for business decisions. Driven by team issues as well as scope definition and cost estimating. Measurable. Cost Research improvements are short term and long term. Continue partnering to fund Cost Research After all, there is an opportunity cost to be paid. We need to spend some amount for Cost Research studies so we are making better planning and programming decisions. Poor cost estimates exacerbate funding problems in the FYDP and throughout the Defense Program Projections. Good cost estimates should lead to better decision making, thus resulting in sound fiscal choices across the total life-cycle. Cost Research really matters as past generations can attest today. I believe we should spend some time looking back at our successes and failures and see if we can see what makes a difference and can we build on the successes to do even better. I’ve been encouraging Rick to provide statistics to show actual program costs compared to CAIG and program office estimates at MS B and see where the chips fall where they will. I’d be willing to co-sponsor research along those lines.

Challenge and Homework Cost Research Symposium 4/15/2018 Challenge and Homework Challenge: Netcentricity goals brings a whole new set of challenges for the cost community. Electronic-intensive systems integrated based on major software development programs. Homework: The cost community needs to place greater emphasis on collection of costs related to both areas. And of course working in terms of capabilities is hard, especially capabilities, like net-centricity and the electronic-intensive and other systems that use a lot of software. To me the scariest words in a program that is having cost growth is—it is only the sofware. Then I know we are in real trouble. These kind of capabilities especially need your expert costing research and skills.

Challenge and Homework Cost Research Symposium 4/15/2018 Challenge and Homework Challenge: The Department established 6 pilot programs to support its Performance Based Logistics (PBL) initiative (MID 917). Current cost accounting systems are not conducive to the collection of platform-specific costs, and our ability to confirm that estimated savings and cost avoidance is actually occurring. Homework: The cost community needs to develop such structures in close coordination with program and budget offices. The cost community needs to work with the logistics community to ensure that the requisite level of visibility into contractor costs is retained. And then there is logistics, often the step-child of the acquisition processes. We need much better visibility into logistics costs, since as you know they are such a large part of the total life cycle cost of our systems and hence of the associated joint capabilities.

Cost Research Symposium 4/15/2018 Now some of you may know my famous (or infamous) cost chart. It’s a good chart—well I might be a little biased—but the programs don’t do a very good job of estimating O&M costs. So another of your challenges is to help us do this better.

Cost Research Symposium 4/15/2018 Inworks to Add a New Spruill "O&S Funding Chart" to Better Support IPTs/DAB Reviews I don’t have the answer but my folks have been thinking about it. Here’s another area that I think would be good for cost research. One that the acquisition community would want to co-sponsor.

Cost Research Symposium 4/15/2018 Closing Thoughts DoD putting more focus on importance the Joint Capability Areas and total life-cycle cost estimates. Our goal is to ensure performance management processes are effective, consistent, and reflective of current industry best practice. We need your continued support to bring about change transition. I remain enthusiastic about the Department’s activities and processes regarding the evolving lexicon of these Joint Capability Areas. I also remain extremely optimistic of the great work you provide each day for the Department. Again, thank you for the opportunity for me to express AT&L’s perspective on the Joint Capability Areas and a host of Cost Research opportunities. This completes part one of my series of nine topics [laugh]. Seriously, I will entertain any questions you might have at this time or when you take a break. Working together is the key to delivering ever-present, credible capability-based cost estimates!

Cost Research Symposium 4/15/2018 Backup

Cost Research Symposium Background Information 4/15/2018 Cost Research Symposium Background Information

Annual Cost Research Symposium 4/15/2018 Annual Cost Research Symposium Since 1989, this Cost Research Symposium has been held every year. 2005 marks the 17th year of the annual Cost Research Symposium. The IDA Cost Research Symposium is jointly sponsored by the OSD CAIG and IDA. ~50 Attendees: By Invite Only Key Cost Managers from the military departments and MDA, FFRDC research managers, DAU reps and a few international cost managers. These folks make decisions on the allocation of cost research funds.

Cost Research Symposium 4/15/2018 Cost Research Background Information

Cost Research Approaches Cost Research Symposium 4/15/2018 Cost Research Approaches Use best available data sources to populate models Actual cost data from earlier program phases CCDRs from analogous programs Recent vintage CERs Historical learning-curve slope realizations Avoid “canned” models Black-box nature of these models inhibits insight Input parameters are too subjective Capture technical and schedule risk through in-depth program knowledge and historical comparisons

Cost Research Symposium Current Cost Estimating Techniques Cost Research Symposium 4/15/2018 High Contractors typically use engineering build-ups Program office methods vary CAIG/CCA use parametric models Contractors use build-ups and vendor quotes PM’s typically use Dem/Val actuals and build-ups CAIG/CCA use parametrics and actuals Cost Estimating Uncertainty Contractors use build-ups, quotes, actuals CAIG/CCA/PM use EMD actuals Low Concept & Tech. Dev. Sys. Dev. & Dem. Production Program phases

Estimate’s Level of Accuracy Cost Research Symposium 4/15/2018 Estimate’s Level of Accuracy Program Success High Estimate’s Level of Accuracy Impact of team’s skills and experience Higher Success ® Level of Scope Definition Program Definition Low ¬ Higher Definition Program Inception Program Completion

Cost Research Symposium 4/15/2018 OSD CAIG Estimates Background Information

Cost Research Symposium 4/15/2018 CAIG Customers SECDEF/DEPSECDEF USD(AT&L): DAB reviews & special studies USECAF: DSABs on Space Programs USD(I)/ASD(NII): Intelligence and C2 programs (Titles 10 & 50) MDA: MOU PA&E: PPBES costs & special studies Others: NASA, USD(C), Congress, OMB AT&L is one of the primary consumers of the cost estimates products produced by the CAIG which may have involved support from most, if not all, of you in attendance today. (NEXT SLIDE)

CAIG Supports Milestones Cost Research Symposium 4/15/2018 CAIG Supports Milestones Develop ICE and Compare to POE/CCP Prepare CAIG Summary Report for OIPT Document Findings and CAIG Estimate/Analysis CAIG Chairman Presents Findings to Committee Work Remaining Issues Pre-Brief D(PA&E) prior to the Defense Acquisition Board (DAB) Coordinate on Draft Acquisition Decision Memo CAIG estimates are used for a variety of critical functions. First, they are used to compare alternative solutions. They also are used to compare life-cycle cost in the analysis of alternate reasonable solutions to a problem. Cost estimates are employed at major milestones to monitor a project and guide decision makers on how to proceed, or whether to proceed, with a program. In addition, cost estimates influence the President’s Budget. Since cost estimates play such a vital statistic of the health and ultimate success of a program, it is critical that the estimates be as accurate as possible; hence the desire to determine current cost and risk models. The CAIG is responsible for providing independent cost estimates of DoD programs and advancing of the DoD’s cost estimation activities. CAIG’s cover all the elements of the total life-cycle cost of a given program include the cost of all research and development efforts, the prime hardware and its major subcomponents, such as support, training, specialized equipment and data, and any military construction. All indications suggest that DoD’s push to leverage joint capabilities is likely to continue. AT&L leadership is continuously examining the effects of DoD transformation. (NEXT SLIDE)

Cost Research Symposium 4/15/2018 Budgeting to CAIG Estimate Budgetary instructions to Services states: “In order to achieve program stability and avoid cost stretch-out, [the Services shall] properly price programs at not less than levels estimated by the Cost Analysis Improvement Group” New guidance has a few “execution” issues: CAIG has been staffed and organized primarily to perform episodic reviews at milestones (and fit in other duties) Continuous maintenance of program estimates generally not done Thus, demand by Services for current CAIG POM numbers often problematic Demand for acquisition milestone reviews continues unabated Final resolution not clear In the short run, CAIG is doing the best it can to provide numbers Expansion of CAIG staff is a possibility, especially given the complexities inherent in producing accurate Joint Capability Area cost estimates As I am rapidly coming to the end of my time, and I would like to leave room for questions. I would like to briefly mention that the shift in DoD planning and operational activities to incorporate a joint focus necessitates the inclusion of new cost factors for determining cost estimations. Investments made on incomplete or inaccurate estimates are likely to run out of funding and, therefore, require justification for additional funding or a reduction of quantity and/or scope. (NEXT SLIDE) AT&L Supports Funding to Best Estimate Based on “Credible” Cost Research!

Cost Research Symposium 4/15/2018 Key Regulatory Costing Events Draft Life-Cycle Cost Estimate (LCCE) Documentation Submission to CAIG/Draft ICE Reviewed w/ PM 45 Days before OIPT Service Cost Position / ICE Review 21 Days before OIPT CAIG Team Formed 180-365 Days Before OIPT CARD Submission to CAIG 180 Days before OIPT Final LCCE Documentation Submission to CAIG 10 Days before OIPT OIPT DAB CARD-Cost Analysis Requirements Description ICE-Independent Cost Estimate OIPT-Overarching Integrated Product Team DAB-Defense Acquisition Board DAE-Defense Acquisition Executive PM-Program Manager OIPT Report Provided to DAE before DAB

Scope of the CAIG Estimates Cost Research Symposium 4/15/2018 Scope of the CAIG Estimates Full Life Cycle Costs- all phases of weapon-system Research Development Test & Evaluation Hardware and Software Supporting Data, Equipment, Training Initial Spares Military Construction (MILCON) Modifications Operations and Support Over System Life Including Manpower Environmental Cost - Compliance, Demilitarization, Clean-Up, Disposal Regardless of Funding Source Based Upon Program Described in the CARD Here are three of the CAIG’s standards for good cost estimates. 1) Completeness of the Full Funding Program Estimate. This includes: All required test assets included Military Construction (MILCON) Provision made for software updates 2) Consistency with Past Experience: Learning curve slopes Engineering Change Order allowance Weight growth 3) Application of Good Costing Practices/Data Account for production rate effects Use actual cost data if available Don’t double count learning and cost reduction measures

Cost Analysis Requirements Description (CARD) Cost Research Symposium 4/15/2018 Cost Analysis Requirements Description (CARD) Description of features pertinent to costing the system Quantitative descriptions of technical, physical, programmatic, and performance characteristics Common definition of system for costing purposes Should be completed 180 days prior to acquisition decision meeting CARD requirements specified in DoD Manual 5000.4-M

Total Life Cycle Cost Estimates Cost Research Symposium 4/15/2018 Total Life Cycle Cost Estimates + Accuracy Range 70% 10% Operations & Support 20% Early in Acquisition Middle in Acquisition Late in Acquisition Life Cycle Costs MS A MS B MS C As depicted in the upper illustration, it becomes much more challenging to effect the kind of changes and our ability to support or alter programs across different program maturities. Early cost Estimates are the basis for pre-acquisition activities, requirements documents, CONOPs, Analysis of Alternatives (AoAs), etc. Common Cost Research provides senior decision makers the basis for business decisions. Business Unit Decisions account for: Asset development strategy Screening of potential projects Commitment of resources The impact of poor cost research estimates results in: Lost opportunities Wasted development effort (dead ends) Lower than expected returns Loss of DoD “credible” cost estimates (e.g., $32B FYDP shortfall) (which I will expand on later) (NEXT SLIDE)

Cost Research Symposium 4/15/2018 Weapon System Timeline and Cost Categories Life Cycle Costs MS A MS B MS C Operating & Support Costs Disposal Costs Investment Costs RDT&E Costs Operations & Support Concept & Technology Development Production & Deployment System Development & Demonstration

Research Accomplishments Cost Research Symposium 4/15/2018 Research Accomplishments Estimate Procedures Measure estimate quality. Correlate estimates with their final cost. Identify estimating improvement areas. Estimate Models Predict accuracy from estimate scores. Best Practices Guide Alignment, process, procedures

OSD Focus Areas for Enhancing Visibility of Operating & Support (O&S) Cost Research Symposium 4/15/2018 OSD Focus Areas for Enhancing Visibility of Operating & Support (O&S) This presentation outlines current thinking on a cost element structure to be used by the military departments for reporting their estimates of operating and support costs for major defense acquisition programs. The Cost Analysis Improvement Group (CAIG) has overall staff oversight responsibility for the collection and analysis of O&S costs for major systems.

OSD CAIG Operating & Support Cost Element Structure (CES) Cost Research Symposium 4/15/2018 OSD CAIG Operating & Support Cost Element Structure (CES) This presentation outlines current thinking on a cost element structure to be used by the military departments for reporting their estimates of operating and support costs for major defense acquisition programs. The Cost Analysis Improvement Group (CAIG) has overall staff oversight responsibility for the collection and analysis of O&S costs for major systems.

Cost Research Symposium 4/15/2018 Big Changes in Cost Cost Analysis Requirements Descriptions (CARDs) Describes system features pertinent to costing Work begins six months before DAB Integrated Product Teams (IPTs) Improves access to costing information Provides problem-solving mechanism Cost estimate structure Detailed estimate at level three or four WBS level No longer high-level parametrics Estimating since early 1990’s

Roles of an O&S Cost Structure Cost Research Symposium 4/15/2018 Roles of an O&S Cost Structure Provides a common structure for cost estimates OSD(CAIG) reviews on MDAPs Creates a common structure for Service cost data “VAMOSC” systems Complements service-unique data structures The structure serves two important purposes. It is a structure that the CAIG uses in reviewing estimates of O&S costs for systems coming before the Defense Acquisition Board. It also provides a structure for the collection of O&S costs under the auspices of the departments’ VAMOSC programs. (VAMOSC = Visibility and Management of Operating and Support Costs) As such, it facilitates the analysis of O&S costs for platforms that from across the departments.

O&S Estimating Structure (Final Draft) Cost Research Symposium 4/15/2018 O&S Estimating Structure (Final Draft) 1.0 Unit Personnel Costs of operators, maintainers, and other support personnel assigned to operating units. Includes military, civilian, and contractor personnel. 2.0 Unit Operations Costs of unit operating material (e.g., fuel and training material), unit support services, and unit travel. Excludes all maintenance and repair material. 3.0 Maintenance Costs of all maintenance other than maintenance personnel assigned to operating units. Includes contractor maintenance. 4.0 Sustaining Support Costs of support activities other than maintenance which can be attributed to a system and are provided by organizations other than operating units. 5.0 Continuing System Improvements Costs of hardware and software modifications to keep the system operating and operationally current. Excludes all costs of modification programs that qualify as Major Defense Acquisition Programs as well as any modifications or upgrades included in the acquisition cost estimate. 6.0 Indirect Support Costs of support activities that provide general services that cannot be directly attributed to a system. Indirect Support is generally provided by centrally managed activities that support a wide range of activities. The CAIG has been working with the departments to simplify the cost element structures used to present O&S cost estimates. The 6 major categories in the current version of the structure are shown here. Some key points: The first two categories aim to capture the costs to operate and support the system at the unit level. The third and fourth categories get at capturing support resources at echelons above the unit. We have proposed a new category – “Continuing System Improvements” – that can be used to capture costs associated with modifications. Finally, the taxonomy includes a category aimed at capturing all the other indirect costs of operating and supporting a system.

O&S Estimating Structure (Final Draft—2nd level) Cost Research Symposium 4/15/2018 O&S Estimating Structure (Final Draft—2nd level) 1. Unit Personnel 1.1 Operations Personnel 1.2 Unit Maintenance Personnel 1.3 Other Direct Support 2. Unit Operations 2.1 Operating Material 2.2 Support Services 2.3 Temporary Duty 3. Maintenance 3.1 Organizational Maintenance 3.2 Intermediate Maintenance 3.3 Depot Maintenance 4. Sustaining Support 4.1 System Specific Training 4.2 Support Equipment Replacement 4.3 Sustaining Engineering and Program Management 4.4 Other Sustaining Support 5. Continuing System Improvements 5.1 Hardware Modifications 5.2 Software Maintenance and Modifications 6. Indirect Support 6.1 Installation Support 6.2 Personnel Support 6.2.1 Personnel Administration (Personnel Acquisition , Individuals Overhead -TPPS) 6.2.2 Personnel Benefits (Family Housing, Commissaries, Child & Family Support, DoD Schools) 6.2.3 Medical Support 6.3 General Training and Education 6.3.1 Basic & Initial Skill Training 6.3.2 Educational Activities This chart lists the second-level cost estimating structure associated with the major categories.

Cost Research Symposium 4/15/2018 Value of Cost Research Improves Current Policy/Guidance Proposes New Policy/Guidance Often Provides Follow-On Actions Improves decision-making at all levels

Cost Research Symposium 4/15/2018 AT&L Efforts to Strengthen the Visibility of the Operating & Support in IPTs/DAB Reviews This presentation outlines current thinking on a cost element structure to be used by the military departments for reporting their estimates of operating and support costs for major defense acquisition programs. The Cost Analysis Improvement Group (CAIG) has overall staff oversight responsibility for the collection and analysis of O&S costs for major systems.

Acquisition Community-Wide Effort Cost Research Symposium 4/15/2018 Acquisition Community-Wide Effort Critical Success Factors Coordination and collaboration across organizational boundaries to provide the best possible support to the acquisition community Collective pursuit of acquisition community transformation vision and goals Unified coalition of acquisition leadership who actively communicate to stakeholders in “one voice” Decision-making from an enterprise-perspective Dedicated, knowledgeable change agents to plan for and execute transformation management activities within their communities Engage acquisition leaders and stakeholders at all levels

Cost Research Symposium 4/15/2018

Cost Research Symposium 4/15/2018 Inworks to Add a New Spruill "O&S Funding Chart" to Better Support IPTs/DAB Reviews

Cost Research Symposium 4/15/2018 AT&L Efforts to Strengthen the Visibility of the Operating & Support in Performance Based Logistics (PBL) This presentation outlines current thinking on a cost element structure to be used by the military departments for reporting their estimates of operating and support costs for major defense acquisition programs. The Cost Analysis Improvement Group (CAIG) has overall staff oversight responsibility for the collection and analysis of O&S costs for major systems.

Cost Research Symposium 4/15/2018 Recent PBL Efforts

Hierarch of Key PBL Publications Cost Research Symposium 4/15/2018 Hierarch of Key PBL Publications

OASD(NII)/DoD Three Goals Background Information Cost Research Symposium 4/15/2018 OASD(NII)/DoD Three Goals Background Information

OASD(NII)/DoD CIO Goals Cost Research Symposium 4/15/2018 OASD(NII)/DoD CIO Goals Goal #1 - Make information available on a network that people depend on and trust. Goal #2 - Populate the network with new, dynamic sources of information to defeat the enemy. Goal #3 - Deny the enemy comparable advantages and exploit weaknesses. ASD(NII)/DoD CIO Home Page: http://www.dod.mil/nii/homepage.html#goals Goal #1 - Make information available on a network that people depend on and trust. Achieve a ubiquitous, secure and robust network Eliminate bandwidth, frequency and computing capability limitations Deploy collaborative capabilities and other performance support tools Populate the network with all Secure and assure the network and the information Goal #2 - Populate the network with new, dynamic sources of information to defeat the enemy data (intelligence, non-intelligence, raw, and processed) Continuously refresh the content of the network Consider that all users of information also are suppliers--post before you process Improve sensemaking Develop new ways to gain access to adversary information Continuously surprise the enemy with the information we are using Goal #3 - Deny the enemy comparable advantages and exploit weaknesses Effectively conduct offensive Information Operations Implement full spectrum security Conduct aggressive counterintelligence Collect persistent, responsive, exquisite, intelligence