1. 1 Jo Ann Lane University of Southern California Center for Systems and Software Engineering jolane@usc.edu Dr. Paul Carlock Northrop Grumman Corporation Mission Systems paul.carlock@ngc.com COCOMO Forum 2006 COSOSIMO: How Well Does It Capture the Unique Aspects of System of Systems Engineering Processes?

2. 2 What is System of Systems Engineering (SoSE) and what makes it unique System of Systems Enterprise Systems Engineering (SoS ESE) and Enterprise Architecture Management Framework (EAMF) Overview Comparison of SoS ESE EAMF with SoSE unique features SoS ESE EAMF track record with respect to SoS success COSOSIMO overview and relationship to SoS ESE EAMF COSOSIMO parameter consistency with respect to SoS ESE key features Conclusions and impact on COSOSIMO evolution Outline

3. 3 SoSE and What Makes it Unique

4. 4 USAF SAB Report on SoSE for Air Force Capability (USAF 2005): The process of planning, analyzing, organizing, and integrating the capabilities of a mix of existing and new systems into a system-of-systems capability that is greater than the sum of the capabilities of the constituent parts. This processes emphasizes the process of discovering, developing, and implementing standards that promote interoperability among systems developed via different sponsorship, management, and primary acquisition processes. National Centers for Systems of Systems Engineering (NCOSOSE): The design, deployment, operation, and transformation of metasystems that must function as an integrated complex system to produce desirable results. These metasystems are themselves comprised of multiple autonomous embedded complex systems that can be diverse in technology, context, operation, geography, and conceptual frame. (http://www.eng.odu.edu/ncsose/what_is_SOSE.shtml)http://www.eng.odu.edu/ncsose/what_is_SOSE.shtml What is SoSE?

5. 5 Key areas where SoSE activities differ from traditional SE –Architecting composability vs. decomposition (Meilich 2006) –Added “ilities” such as flexibility, adaptability, composability (USAF 2005) –Net-friendly vs. hierachical (Meilich 2006) –First order tradeoffs above the component systems level (e.g., optimization at the SoS level, instead of at the component system level) (Garber 2006) –Early tradeoffs/evaluations of alternatives (Finley 2006) –Human as part of the SoS (Siel 2006, Meilich 2006, USAF 2005) –Discovery and application of convergence protocols (USAF 2005) SoSE Compared to Traditional SE Activities

6. 6 Key areas where SoSE activities differ from traditional SE (continued) –Organizational scope defined at runtime instead of at system development time (Meilich 2006) –Dynamic reconfiguration of architecture as needs change (USAF 2005) –Modeling and simulation, in particular to better understand “emergent behaviors” (Finley 2006) –Component systems separately acquired and continue to be managed as independent systems (USAF 2005) –Intense concept phase analysis followed by continuous anticipation; aided by ongoing experimentation (USAF 2005) SoSE Compared to Traditional SE Activities (continued)

7. 7 Key Challenges for SoSE –Business model and incentives to encourage working together at the SoS level (Garber 2006) –Doing the necessary tradeoffs at the SoS level (Garber 2006) –Human-system integration (Siel 2006, Meilich 2006) –Commonality of data, architecture, and business strategies at the SoS level (Pair 2006) –Removing multiple decision making layers (Pair 2006) –Requiring accountability at the enterprise level (Pair 2006) –Evolution management (Meilich 2006) –Maturity of technology (Finley 2006) For the most part, SoSE appears to be SE+ SoSE Compared to Traditional SE Activities (continued)

8. 8 SoS ESE Overview

9. 9 SoS ESE “System of Systems (SoS) Enterprise Systems Engineering (ESE) (also called “Agency-Level Systems Engineering” for federal enterprises) is the set of processes and activities devoted to capability-delivery design and integration throughout an enterprise’s mission planning. It translates and implements the enterprise’s goals and objectives into a comprehensive and coherent Enterprise Architecture, a Strategic Plan for “System of Systems” Evolution or transformation. Essentially, it is the enterprise’s Strategic Planning and Control Process.” SoS ESE was developed to respond to the Information Technology Management Reform Act of 1996 and the Government Performance and Results Act (GPRA) of 1993. Low risk transformation of complex SoS Enterprises requires the discipline and rigor of SoS ESE. SoS ESE has a solid technical foundation that satisfies federal mandates for enterprise strategic planning and control. By way of definition…

10. 10 The Enterprise Architecture (EA) incorporates both technical and programmatic information to – Define the strategic plan and roadmap for transformation – Support informed investment decision making The Enterprise Architecture Management Framework (EAMF) provides the disciplined processes to evolve, maintain, and ensure the proper implementation of the Enterprise Architecture both efficiently and successfully – Combined focus on strategic, tactical, and operational objectives – Clear roles and responsibilities are assigned to each partner in agency transformation – Governance support mechanisms for managing transformation activities Integrates and leverages Best Practices from successful large-agency transformation efforts already accomplished An approach to managing enterprise transformation that n Utilizes a comprehensive, integrated, mission service-based enterprise architecture n Defines an enterprise architecture management strategy that effectively employs all elements of the acquisition framework to minimize risk and expedite delivery of benefits n Is flexible to accommodate and leverage proven best practices of mature corporate, mission, and cultural process reengineering methodologies The SoS ESE Methodology Architecting for Enterprise-Level Transformation

11. 11 Three-Level SoS ESE Methodology Tight linkage to the Organization’s Acquisition Strategy is critical to establishing a transformation partnership

12. 12 Integrated technical and programmatic description of the enterprise Includes technical, cost, and schedule data, and all other data necessary to define a strategic plan – Supports development of all architecture views (DoDAF, business, temporal, security, data, cost, …) Lays out the transition of capabilities aligned with agency and stakeholder values and priorities – Mission service-based derived from community needs Mission service provision through capabilities Capability provision through systems, people, facilities, support activities Temporal in nature – Capability provision “how” changes with time The enterprise architecture IS the strategic plan for enterprise transformation Characterizing the Enterprise Architecture (EA) DoDAF Views

13. 13 Characterizing the Enterprise Architecture Management Framework Seamless life cycle acquisition management process that extends from identification of need to capability retirement

14. 14 The EAMF ensures proper implementation of the enterprise architecture by explicitly integrating systems engineering into the acquisition process Enterprise Architecture Management Framework (EAMF)

15. 15 Enterprise Architecture Taxonomy The enterprise architecture balances User Community needs against resources to achieve successful transformation

16. 16 Industry Best Practices in the SoS ESE Methodology

17. 17 Performance Modeling and Measurement System Needs and Requirements Management Proven SE Provides the Foundation n Requirements Analysis n Architecture Development n Technology Insertion n Risk Management n CM n IV&V n Readiness n … Systems Engineering Activities

18. 18 Value Engineering ensures a customer focus and tight alignment of enterprise strategic planning and implementation Value Engineering Provides the Strategic Edge SE transformation discipline – Quick, decisive performance improvements – Aligns strategy with capabilities – Builds ownership and acceptance Key areas: – Strategy development – Capability analysis – Change management – Communications

19. 19 Mission Engineering comprehensively defines capabilities to support effective systems engineering Mission Engineering Enables Dynamic Responsiveness SE discipline for visual definition and linkage to requirements – Transforms capabilities into requirements – Captures Mission and business processes Business rules Design approaches – Vehicle for collaborative hand-off among performing organization elements Key activities – Community requirements – Capability requirements – Systems requirements Enterprise Activity Roadmap (Operations Analysis) Multi-Dimensional Requirements View

20. 20 Melding of proven SE disciplines provides a rapid Enterprise Architecture definition Application Within the EAMF Integrated disciplines provide critical artifacts across the EAMF to update EA – Tight interface to customers and priorities – Flexible for varying situations – Responsive to short timeline needs Provides the basis for a decision support framework – Technical and programmatic data – All stakeholders involved

21. 21 SoS ESE EAMF Track Record Successes –FAA (1998-2001): Led to Congress trusting the FAA architecture planning process and increasing the budget by 42% (9/11 changed priorities) –National System for Geospatial-Intelligence: Using to guide enterprise architecture development. Incorporates scenario-based engineering into the framework. –Other classified programs Reasons why ESE EAMF has not worked well on some other programs –Lack of senior management commitment to processes –Interdependencies not accurately reported and tracked –Inaccurate tracking of schedule/budget issues The EA, as a strategic plan, is only as good as the data it contains and the commitment of the organization to keep it current and correct

22. 22 COSOSIMO Overview and Relationship to SoS ESE EAMF

23. 23 Planning, Requirements Management, and Architecting (PRA) Source Selection and Supplier Oversight (SO) SoS Integration and Testing (I&T) Size Drivers Cost Drivers SoS Definition and Integration Effort COSOSIMO Reduced Parameter Sub-Model Overview

24. 24 COSOSIMO/SoS ESE EAMF Relationship PRA COSOSIMO SOI&T Strategic Analysis Capability Analysis Alternative Analysis Implement Operations And Sustainment

25. 25 COSOSIMO Parameter Consistency With Respect to SoS ESE Key Features

26. 26 PRA Size Drivers # SoS-related requirements # SoS interface protocols Cost Drivers Requirements understanding Level of service requirements Stakeholder team cohesion SoS team capability Maturity of LSI processes Tool support Cost/schedule compatibility SoS risk resolution LSI PRA Effort Associated EAMF Key Features Tight interface to customers and priorities Flexible for varying situations Responsive to short timelines needs Provides the basis for a decision support framework Integrated technical and programmatic (including cost) data All stakeholders involved COSOSIMO: PRA Sub-Model

27. 27 SO Size Drivers # independent component system organizations Cost Drivers Requirements understanding Architecture maturity Level of service requirements SoS team capability Maturity of LSI processes Tool support Cost/schedule compatibility SoS risk resolution LSI SO Effort Associated EAMF Key Features Identification of viable alternatives Comprehensive analysis of alternatives Selection based on stakeholder priorities Consistency with both current and projected budget profiles COSOSIMO: SO Sub-Model

28. 28 I&T Size Drivers # SoS interface protocols # SoS scenarios # unique component systems Cost Drivers Requirements understanding Architecture maturity Level of service requirements SoS team capability Maturity of LSI processes Tool support Cost/schedule compatibility SoS risk resolution Component system maturity and stability Component system readiness LSI I&T Effort Associated EAMF Key Features Customer communications and feedback IV&V Transformation assessment and impact review In-service metrics and feedback COSOSIMO: I&T Sub-Model

29. 29 Conclusions and Impact on COSOSIMO Evolution

30. 30 Summary What is “special” about SoSE in the enterprise environment? –SoSE is SE+ with the focus being on “enterprise transformation” –SoSE is SE tightly integrated with the acquisition process –SoSE requires flexibility, flexibility, and more flexibility in both engineering and acquisition as the environment changes during the transformation process –SoSE requires more of a “governance support” mechanism than long term detailed planning and structured oversight –Key features to being successful Need a Strategic Plan/Enterprise Architecture for on-going SoS evolution and transformation that includes technical, cost, and schedule aspects Planning and honesty about variance in actual progress are critical—need to constantly adjust to reality A flexible, evolvable architecture is required for on-going success Understanding current business processes and re-engineering those processes to take advantage of SoS capabilities is key

31. 31 Summary (continued) How does EAMF compare to SoSE observations? –Addresses many of the differences and challenges identified in recent conferences and workshops: Added “ilities” such as flexibility, adaptability, composability Early tradeoffs and tradeoffs above the component systems level Managerial independence of component systems Intense concept phase analysis followed by continuous anticipation Business model and incentives to encourage working together at the SoS level Commonality of data, architecture, and business strategies at the SoS level Requiring accountability at the enterprise level Evolution management Technology maturity

32. 32 Summary (continued) Can “close enough” SoSE effort estimates be obtained from current SE cost models such as COSYSMO? –Probably not –Current SE cost models do not account for levels of complexities seen in many SoSs, the need for significant business process re- engineering, and the coordination of multiple component system “owners” –Still need data from SoSE programs to determine the extent and impact of these differences What does this mean for COSOSIMO? –There is an identified need for COSOSIMO—those that are in the midst of SoSE programs do not have the estimation support they need from existing cost models –The path taken will be determined by the number and types of differences from the current SE cost model, COSYSMO

33. 33 Carlock, P.G., and R.E. Fenton, "System of Systems (SoS) Enterprise Systems for Information- Intensive Organizations," Systems Engineering, Vol. 4, No. 4, pp. 242-261, 2001 DiMario, Mike (2006); “System of Systems Characteristics and Interoperability in Joint Command Control”, Proceedings of the 2nd Annual System of Systems Engineering Conference Electronic Industries Alliance (1999); EIA Standard 632: Processes for Engineering a System Finley, James (2006); “Keynote Address”, Proceedings of the 2nd Annual System of Systems Engineering Conference Garber, Vitalij (2006); “Keynote Presentation”, Proceedings of the 2nd Annual System of Systems Engineering Conference INCOSE (2006); Systems Engineering Handbook, Version 3, INCOSE-TP-2003-002-03 Krygiel, A. (1999); Behind the Wizard’s Curtain; CCRP Publication Series, July, 1999, p. 33 Maier, M. (1998); “Architecting Principles for Systems-of-Systems”; Systems Engineering, Vol. 1, No. 4 (pp 267-284) Meilich, Abe (2006); “System of Systems Engineering (SoSE) and Architecture Challenges in a Net Centric Environment”, Proceedings of the 2nd Annual System of Systems Engineering Conference Pair, Major General Carlos (2006); “Keynote Presentation”, Proceedings of the 2nd Annual System of Systems Engineering Conference Proceedings of AFOSR SoSE Workshop, Sponsored by Purdue University, 17-18 May 2006 Proceedings of Society for Design and Process Science 9 th World Conference on Integrated Design and Process Technology, San Diego, CA, 25-30 June 2006 Siel, Carl (2006); “Keynote Presentation”, Proceedings of the 2nd Annual System of Systems Engineering Conference United States Air Force Scientific Advisory Board (2005); Report on System-of-Systems Engineering for Air Force Capability Development; Public Release SAB-TR-05-04 SoSE References