1. Program Analysis & Evaluation 1 © 2006 July 13, 2015 11/18/98 2:13 PM Research Sponsors Robert Flowe, Gary Bliss OSD Program Analysis and Evaluation, Resource Analysis Capabilities Based Cost Analysis Adapting to a New Paradigm Overview of an OSD-Sponsored Research Project* Principal Researchers Dr. Maureen Brown, Mr. Sean Hamel University of North Carolina School of Government Dr. David Zubrow, Mr. Bill Anderson, Mr. Jim McCurley Software Engineering Institute Mr. Robert Jones, Ms. Elizabeth Koza, Mr. John Wilke, Mr. Paul Hardin Technomics, Inc. Dr. David Usechak OSEC Supporting Co-Sponsors Office of the Under Secretary of Defense (Acquisition Technology & Logistics) Air Force Cost Analysis Agency Office of the Deputy Assistant Secretary of the Army for Cost and Economics *The views expressed in this presentation are those of the author, and should not be construed as representing the views of the Department of Defense, or its components

2. Program Analysis & Evaluation 2 © 2006 July 13, 2015 11/18/98 2:13 PM AGENDA Background  Capabilities & Portfolio Management (C/PM) Issues  Interdependence and Outcomes Hypothesis Risk Indicators – Software Engineering Institute  Interdependence and Acquisition Risk Interdependence – University of North Carolina  Programmatic Interdependence and Cost/Schedule Breaches Architectural Parametric Modeling – Technomics  Constructive Interdependence and Development Resource Demands Next Steps

3. Program Analysis & Evaluation 3 © 2006 July 13, 2015 11/18/98 2:13 PM Three “Domains” of Interdependence Programmatic - Acquisition and management practices Constructive – Technical construction activities Operational – Mission, goals, objectives, and their fulfillment programmatic operational constructiveconstructive

4. Program Analysis & Evaluation 4 © 2006 July 13, 2015 11/18/98 2:13 PM Relate to “Induced Costs” Relate to “Inherent Costs” How Does ‘Interdependence’ Matter? Concept Map of Attributes and Outcomes Cost Schedule Performance (Outcomes) Drives Total Effort Inherent Effort Induced Effort Constructive Interdependence Programmatic Interdependence

5. Program Analysis & Evaluation 5 © 2006 July 13, 2015 11/18/98 2:13 PM C/PM vs “Big”– Management Issues Capability and portfolio management (C/PM) drives cooperative management of entities which are developed and managed separately This creates induced costs due to (for example)  decentralized management and independent authority of constituent systems  asymmetrical incentives for SoS vs constituent objects/systems goals and objectives  different maturities of the objects/systems C/PM incurs effects beyond scale alone

6. Program Analysis & Evaluation 6 © 2006 July 13, 2015 11/18/98 2:13 PM What’s Wrong With Current Methods? Department management and oversight processes still largely program-centric Interdependence effects not routinely captured Transaction costs largely exogenous to program baselines Failure to understand effects of interdependence  Results in unanticipated, uncontrolled program cost & schedule growth  Results in diminished capacity to develop joint capabilities

7. Program Analysis & Evaluation 7 © 2006 July 13, 2015 11/18/98 2:13 PM Diagnostic Risk Indicators Inter-program interdependence influences each program’s outcomes  suggests potential risk areas for program execution A risk taxonomy  applicable to prospective SoS or joint capability implementations Early insight  allow proactive mitigation

8. Program Analysis & Evaluation 8 © 2006 July 13, 2015 11/18/98 2:13 PM Systems-of-Systems Risk Taxonomy

9. Program Analysis & Evaluation 9 © 2006 July 13, 2015 11/18/98 2:13 PM In Systems-of-Systems, the requirement for interoperability expands the Risk Taxonomy

10. Program Analysis & Evaluation 10 © 2006 July 13, 2015 11/18/98 2:13 PM Interoperability Expands the Risk Taxonomy

11. Program Analysis & Evaluation 11 © 2006 July 13, 2015 11/18/98 2:13 PM Observations of Joint Capabilities & Interdependence No program is an “Island” Programs share and transfer resources across programmatic boundaries These “resources” include:  Capital  Materiel  Labor  Authority  Data

12. Program Analysis & Evaluation 12 © 2006 July 13, 2015 11/18/98 2:13 PM Observation: Interdependence Correlates to Schedule Slippage

13. Program Analysis & Evaluation 13 © 2006 July 13, 2015 11/18/98 2:13 PM Parametric Model: Study of Architectural Based Cost Estimation Hypothesis: a relationship exists between resources required and architecture-based interdependencies

14. Program Analysis & Evaluation 14 © 2006 July 13, 2015 11/18/98 2:13 PM Preliminary Results A Parametric Estimating Relationship provides preliminary evidence that there is a relationship between resources required and architecture-based interdependencies * The RDT&E Budget, i.e., the sum of actual and estimated costs to completion, is used as a proxy for RDT&E Cost Equation (1) *

15. Program Analysis & Evaluation 15 © 2006 July 13, 2015 11/18/98 2:13 PM Parametric Model: Study of Architectural Based Cost Estimation Results: Preliminary analysis using Equation (1) supports the hypothesis that a relationship exists between resources required and architecture-based interdependencies Next Step: Apply analysis approach to programs using costs instead of budgets

16. Program Analysis & Evaluation 16 © 2006 July 13, 2015 11/18/98 2:13 PM Next Steps Technical Research  Pilot Study Model detailed relationships Develop representation of overall program properties Validate data gathering & analysis methods  Risk Indicators Continuing analysis for confirmation of risk indicators  Interdependence Cost and Schedule Consequences Expand data points and refine Cost Estimating Relationships Explore portfolio analysis for targets of interest Collaborations  Continue collaborative work with Services & AT&L  Seek collaboration with ASD/NII on DoDAF  Develop collaborations with T&E community for test data