Presentation is loading. Please wait.

Presentation is loading. Please wait.

© 2013 PRICE Systems, LLC All Rights Reserved | Decades of Cost Management Excellence 1 Data Collection and Analysis Supporting Defendable Cost Estimates.

Similar presentations


Presentation on theme: "© 2013 PRICE Systems, LLC All Rights Reserved | Decades of Cost Management Excellence 1 Data Collection and Analysis Supporting Defendable Cost Estimates."— Presentation transcript:

1 © 2013 PRICE Systems, LLC All Rights Reserved | Decades of Cost Management Excellence 1 Data Collection and Analysis Supporting Defendable Cost Estimates Arlene Minkiewicz, Chief Scientist PRICE Systems, LLC

2 © 2013 PRICE Systems, LLC All Rights Reserved | Decades of Cost Management Excellence 2 Agenda Introduction Historical Prospective on Cost Estimation Data Driven Estimation and its Importance in the current economic client Pilot Case Studies – Software Data Collection – Automotive Data Collection and Analysis Lessons Learned

3 © 2013 PRICE Systems, LLC All Rights Reserved | Decades of Cost Management Excellence 3 Introduction Cost estimation in the Aerospace and Defense world emerged as a necessary around the time of World War II All sorts of mathematical and experiential models have been applied to the problem with varying degrees of success More and more organizations are pushing for models tuned to their specific data and experience Many of these organizations lack the infrastructure, processes and tools to collect and analyze project data efficiently This paper presents two pilot projects (both on-going) to inject data driven estimation into two very different organizations.

4 © 2013 PRICE Systems, LLC All Rights Reserved | Decades of Cost Management Excellence 4 Historical Perspective General H.H. Hap Arnold at the end of WWII – During the war, the Army, Army Air Forces, and the Navy have made unprecedented use of scientific and industrial resources. The conclusion is inescapable that we have not yet established the balance necessary to insure the continuation of teamwork among the military, other government agencies, industry, and the universities. Scientific planning must be years in advance of the actual research and development work.

5 © 2013 PRICE Systems, LLC All Rights Reserved | Decades of Cost Management Excellence 5 Historical Perspective Leaders took this to heart – state of the art weapons systems are necessary to ensure military superiority Beginning of period of time during which the Department of Defense (DOD) is plagued with issues surrounding cost!

6 © 2013 PRICE Systems, LLC All Rights Reserved | Decades of Cost Management Excellence 6 Historical Perspective PRICE H as an RCA Only Tool Operations Research applied to Military Affairs Proliferation of Commercially Available Cost Estimation Models Great Depression World War II Severe DoD Project Cost Overruns OSD CAIG Formed War on Terrorism Berlin Wall Breached RAND Created PMI Created PC Project Mgmt EVM Tools Parametric Estimating Initiative ISPA Founded Korean ISPA Chapter Defense Transformation Operations Research and Statistics applied to Industrial Affairs Homeland Defense Govt & Industry Business Modernization Cost Overruns J. Gordon Davis Frank Frieman

7 © 2013 PRICE Systems, LLC All Rights Reserved | Decades of Cost Management Excellence 7 Data Driven Estimation Not a new idea – just a new focus on whats behind an estimate More credible and lends a a sense of defendability to any estimate Data-driven estimation IS…

8 © 2013 PRICE Systems, LLC All Rights Reserved | Decades of Cost Management Excellence 8 Data Driven Estimation Based on data extrapolation from analogous systems Based on a model using analogous systems to determine model inputs Based on a model calibrated to historical data A data-driven estimate MAY BE…

9 © 2013 PRICE Systems, LLC All Rights Reserved | Decades of Cost Management Excellence 9 Data Driven Estimation Transparent as to what data was used Obtained from a credible, identifiable source Clear about how that data was used A data-driven estimate MUST BE…

10 © 2013 PRICE Systems, LLC All Rights Reserved | Decades of Cost Management Excellence 10 Case Study Pilot – Software Data Collection Software data collection and analysis to support more defendable estimate – Army Program Executive Office for Simulation, Training, and Instrumentation (PEO STRI) – No data collection processes in place – Pilot involves one large, complex program – two contractors – one does primarily development, one primarily integration

11 © 2013 PRICE Systems, LLC All Rights Reserved | Decades of Cost Management Excellence 11 Software Pilot - Motivation PEO STRI personnel take their mission of delivering training to the warfighter seriously and intend to deliver best of breed solutions. Estimates presented to the Office of the Deputy Assistant Secretary of Defense for Cost and Economics (ODASA-CE) are met with the question Wheres the data that supports this estimate?

12 © 2013 PRICE Systems, LLC All Rights Reserved | Decades of Cost Management Excellence 12 Software Pilot - Motivation Unsatisfactory responses inspire large risk margins resulting in less budget available to the program Furthermore, without insight into projected costs for specific capabilities, it is difficult to make trade-offs to accomplish reduced costs PEO STRI wants to establish a consistent, repeatable process to collect cost and technical data for each capability they develop in their programs

13 © 2013 PRICE Systems, LLC All Rights Reserved | Decades of Cost Management Excellence 13 Data Collection – How will it be used

14 © 2013 PRICE Systems, LLC All Rights Reserved | Decades of Cost Management Excellence 14 Reporting Strategy for Maximum Benefit Report

15 © 2013 PRICE Systems, LLC All Rights Reserved | Decades of Cost Management Excellence 15 Data Collection and Normalization Once the need for data collection has been identified, an important first step is identifying what to collect. This requires – Understanding the goal of the collection program – Understanding what data is necessary to support that goal – Understanding what data is available or could reasonably be made available Normalization guidelines need to be developed The next step needs to be documentation of these understandings and guidelines in a clear, concise, understandable and repeatable fashion. Automation eases the collection process, ensures consistency in the process and facilitates the process of normalization

16 © 2013 PRICE Systems, LLC All Rights Reserved | Decades of Cost Management Excellence 16 Data Collection – Keys to Success Use of an Integrated Product Team (IPT) Data Item Dictionary created to define…. – What data to collect – A clear understanding of the data and potential uses – How often to collect it – How to align project specific data to more general categories (across multiple contractors) PEO STRI PRICE Contractors Lots of collaboration between PEO STRI personnel, contractors and the PRICE team resulting in…..

17 © 2013 PRICE Systems, LLC All Rights Reserved | Decades of Cost Management Excellence 17 Data Collection – Keys to Success Requirement added to Contract Deliverable Requirements List (CDRL) making data collection a contractor requirement Creation of automated code counting tools by wrapping USCs Code Counting tool with Excel to help ensure consistency of data Software Resource Report (SRR) developed to align with output of code counting tool, automate mapping processes and align with potential uses Lots of collaboration between PEO STRI personnel, contractors and the PRICE team resulting in…..

18 © 2013 PRICE Systems, LLC All Rights Reserved | Decades of Cost Management Excellence 18 Software Pilot – Software Resource Report

19 © 2013 PRICE Systems, LLC All Rights Reserved | Decades of Cost Management Excellence 19 PEO STRI Data Collection and Preparation Process Collect OneSAF Program data using SRR forms. Data collected includes: – Code counts (New, Modified, Deleted, Unmodified) – Effort by Activity – Effort by Capability – Factors relevant to project, team & iteration Extract data and import into TrueFindings format Create TruePlanning® Product Breakdown Structure (PBS) by: – Contractor – Build – Capability

20 © 2013 PRICE Systems, LLC All Rights Reserved | Decades of Cost Management Excellence 20 PEO STRI Data Collection and Preparation Process Calibrate the inputs of interest in TruePlanning® using data from TrueFindings – Organizational Productivity – Functional Complexity by capability by language Import the calibrated inputs into TrueFindings file Drive inputs values using the findings from TrueFindings inside TruePlanning® to create defendable estimates.

21 © 2013 PRICE Systems, LLC All Rights Reserved | Decades of Cost Management Excellence 21 Software Pilot – Progress to Date Data collection occurs every 10 weeks. The first iteration was a thrown away – Automation failures in the contractor configuration – Uncertainty of some data definitions – Additional discussions required by IPT to iron out questions and uncertainties Four successful iterations have occurred since then – each resulting in slight refinement to the process, DID and/or SRR A calibration process has been created and continues to be refined Collected data and calibration results have been integrated into a TrueFindings database to support future estimation efforts

22 © 2013 PRICE Systems, LLC All Rights Reserved | Decades of Cost Management Excellence 22 Data Collection - Lessons Learned Alleviate contractor concern about being measured or misuse of data Facilitate determination of data collection targets Need buy in from the data collectors Entire team needs to understand motivation for data collection

23 © 2013 PRICE Systems, LLC All Rights Reserved | Decades of Cost Management Excellence 23 Data Collection - Lessons Learned Creates consistency Creates efficiency Reduces impact on contractors Promotes harmony Only means to handle such large code counts / bases Facilitates normalization and enforces normalization guidelines Automation is key to success

24 © 2013 PRICE Systems, LLC All Rights Reserved | Decades of Cost Management Excellence 24 Data Collection - Lessons Learned Communication, teamwork and building trust are paramount Data needs to be inspected for quality and ported into an environment for easy analysis.

25 © 2013 PRICE Systems, LLC All Rights Reserved | Decades of Cost Management Excellence 25 Conclusions The need for data driven estimation is driving many organizations in Aerospace and Defense and industry in general to look for new ways to collect and use data Data driven estimation is really nothing new – we have been using data (real or experiential) to justify our estimates forever. The new requirement is transparency and openness about the data and how it is being used. To support this effort organizations may need to establish formal processes for collecting their project data. When doing this it is important to understand the importance of – Communication – Automation – Flexibility – Buy in across all the stakeholders

26 © 2013 PRICE Systems, LLC All Rights Reserved | Decades of Cost Management Excellence 26 Questions

27 © 2013 PRICE Systems, LLC All Rights Reserved | Decades of Cost Management Excellence 27 © 2013 Financial Executives International Backup Slides Data Collection and Analysis Supporting Defendable Cost Estimates Arlene Minkiewicz, Chief Scientist PRICE Systems, LLC


Download ppt "© 2013 PRICE Systems, LLC All Rights Reserved | Decades of Cost Management Excellence 1 Data Collection and Analysis Supporting Defendable Cost Estimates."

Similar presentations


Ads by Google