1. System of Systems: What They Are and How to Engineer Them
Jo Ann Lane
jolane at usc.edu
12 April 2010

2. Overview SoS context and key challenges SoSE strategies SoS example
Incremental commitment and evolution
Lean principles
Engineering cost estimation
Engineering and management artifacts
Test and evaluation
SoS example
Regional Area Crisis Management System (RACRS)
Future plans
Acknowledgements

3. What is a “System of Systems”?
Very large systems using a framework or architecture to integrate constituent systems (CSs)
Exhibits emergent behavior not otherwise achievable by CSs
SoS CSs
Independently developed and managed
New or existing systems in various stages of development/evolution
May include a significant number of COTS products
Have their own purpose
Can dynamically come and go from SoS
Typical domains
Military/Crisis Response: Dynamic communications infrastructure
Business: Enterprise-wide and cross-enterprise integrations
Net -
Centric SoS
Net-Centric
Connectivity
Laboratory
System
Imaging
Management
Pharmacy
Patient
Telemetry
Health Care Network
Based on Mark Maier’s SoS definition [Maier, 1998]

4. SoS Taxonomy Virtual [Maier, 1998] Collaborative [Maier, 1998]
Lacks a central management authority and a clear SoS purpose
Collaborative [Maier, 1998]
CS engineering teams work together, but no overarching SoSE team to guide
Acknowledged [Dahmann, 2008]
Have recognized objectives, a designated manager, and resources at the SoS level (SoSE team)
Directed [Maier, 2008]
SoS centrally managed by a government, corporate, or Lead System Integrator (LSI) and built to fulfill specific purposes

5. Example: SoSE (Directed)
Source
Selection ●
● ●
Valuation
Exploration
Architecting
Develop
Operation
System A
System B
System C
System x
LCO-type
Proposal &
Feasibility
Info
Candidate Supplier/
Strategic Partner n
Strategic Partner 1
SoS-Level
FCR1
DCR1
OCR1
Rebaseline/
Adjustment FCR1
OCR2
  
OCRx1
FCRB
DCRB
OCRB1
FCRA
DCRA
FCRC
DCRC
OCRC1
OCRx2
OCRx3
OCRx4
OCRx5
OCRC2
OCRB2
OCRA1 5

6. Example: SoSE (Acknowledged)
Source
Selection?
Existing
collaborative
SoS? ●
● ●
Valuation
Exploration
Architecting
Develop
Operation
System A
System B
System C
System x
Candidate Supplier/
Strategic Partner n
Strategic Partner 1
SoS-Level
FCR1
DCR1
OCR1
Rebaseline/
Adjustment FCR1
OCR2
  
OCRx1
FCRB
DCRB
OCRB1
FCRA
DCRA
FCRC
DCRC
OCRC1
OCRx2
OCRx3
OCRx4
OCRx5
OCRC2
OCRB2
OCRA1 6

7. Example: SoSE (Collaborative)
Rebaseline/
Adjustment FCR1
FCR1
DCR1
OCR1
OCR2
SoS-Level
Exploration
Valuation
Architecting
Develop
Operation
  
OCRx1
OCRx2
OCRx3
OCRx4
OCRx5
System x
  
Develop
Operation
Operation
Operation
Operation
   ●
● ●
FCRC
DCRC
OCRC1
OCRC2
System C
  
Exploration
Valuation
Architecting
Develop
Operation
  
FCRB
DCRB
OCRB1
OCRB2
System B
  
Exploration
Valuation
Architecting
Develop
Operation
  
FCRA
DCRA
OCRA1
System A
  
Exploration
Valuation
Architecting
Develop
Operation
   7

8. SoSE Activities and Challenges for “Acknowledged” SoS
Translating
capability
objectives
Addressing new
requirements
& options
Addressing
& solution
options
Understanding
systems &
relationships
(includes plans)
External Environment
Developing,
evolving and
maintaining
SoS design/arch
Developing
& evolving
SoS
architecture
Assessing
(actual)
performance
to capability
Orchestrating
upgrades
to SoS
Monitoring
& assessing
changes
SoSE Guidebook view based on interviews and analysis of 18 DoD SoSs in various stages:
Communications systems
Command and control systems
Integrated combat systems
Ballistic missile defense systems
Intelligence information systems
Space-related systems
Key challenges
Focusing CSs on SoS needs and capabilities
Coordinating development of new capabilities across CSs
Creating SoS roadmap to guide CS activities
Testing SoS capabilities in an asynchronous development environment

9. SoSE Core Element Description
Translating Capability Objectives
Starts with an SoS need or new capability
Works to understand new capability and alternatives for providing it
Understanding Systems and Their Relationships
Collects and maintains information about current state of the SoS and its CSs
Assessing Performance to Capability Objectives
Evaluation of current performance and how performance meets current and future needs
Developing/Evolving SoS Architecture
Evaluation of existing SoS architecture and identification of alternatives to mitigate limitations and improve performance
Monitoring and Assessing Changes
Monitoring of CS non-SoS changes
Addressing Requirements and Solution Options
Evaluation/prioritization of SoS reqs
Evaluation of solution options and selection of option
Orchestrating Upgrades
Oversight activity to monitor progress of the CS SoS capability upgrades and mitigate obstacles

10. Current System Acquisition Methods Easy to misinterpret as one-size-fits-all
V-Model1
Spiral Model2
High level guidance assumes that acquirers have extensive acquisition experience...
Without experience, too easy to misinterpret and auger in with disastrous results...

11. Typical Acquisition Process
Aircraft pilot coming off a cargo plane is assigned to manage/ oversee the acquisition of a new aircraft subsystem
Excellent understanding of aircraft personnel needs
No experience with system/software development
Conditioned to plan the flight and fly the plan
Tends to interpret V-model diagram sequentially
Tends to interpret spiral diagram as one-size-fits-all
Leading to
Excessive complexity and confusion
Missed budgets and schedules

12. Typical IT Acquisition/Legacy Replacement Process
Organization has own IT department to handle maintenance and evolution of existing system
New system requires additional resources
Search for COTS solution
Outsource development of new system
Increased risks due to
Lack of detailed knowledge about existing IT systems
Little/no experience with COTS vendor or outsource supplier

13. Additional Systems and SoS Future Challenges
Rapid pace of change
In competition, mission priorities, technology, Commercial Off-the-Shelf (COTS), environment
Need incremental development to avoid obsolescence
Need concurrent vs. sequential processes
Need both prescience and rapid adaptability
Software important; humans more important
Brownfield vs. Greenfield development
Need to provide legacy continuity of service
Need to accommodate legacy, OTS constraints
Always-on, never-fail systems
Need well-controlled, high-assurance processes
Need to synchronize and stabilize concurrency
Need to balance assurance and agility

14. Rapid Change Creates a Late Cone of Uncertainty – Need incremental vs
Rapid Change Creates a Late Cone of Uncertainty – Need incremental vs. one-shot development
Uncertainties in competition, technology, organizations, mission priorities
There is Another Cone of Uncertainty: Shorter increments are better
Uncertainties in competition and technology evolution and changes in organizations and mission priorities, can wreak havoc with the best of system development programs. In addition, the longer the development cycle, the more likely it will be that several of these uncertainties or changes will occur and make the originally-defined system obsolete. Therefore, planning to develop a system using short increments helps to ensure that early, high priority capabilities can be developed and fielded and changes can be more easily accommodated in future increments. 14

15. SoSE Synchronization Challenges
SoS SE Level*
Constituent System n
(pre-existing)
  
Increment m
Increment m+1
   ●
● ●
Constituent
System B
(pre-existing)
  
Increment n-1
Increment n
Increment n+1
  
MS A
MS B
MS C
New System A
  
MSA TD
EMD PD
O&S
   15

16. SoSE Process Strategies: Incremental Commitment Model for SoS
Clear “battle rhythm” for SoS incremental upgrades, driven by prioritized backlog of needed capabilities….
Constituent systems use their own lifecycle upgrade processes to integrate SoS requirements into their own incremental upgrade….

17. Initial Lean Indicators in SoSE
Holistic view across SoS, focus on long term goals
Guided by prioritized stakeholder capability needs
Continuous learning organization
Monitoring and assessing changes
Assessing performance to capability objectives
SoSE “battle rhythm”
Make decisions slowly by consensus
Consider all options
Continuous process flow through concurrent engineering
“Pull” CS engineering knowledge as needed
Implement rapidly
Level the workload—use systems with available “bandwidth”
Minimize waste by
Avoiding duplication of engineering efforts at CS level
Eliminating none-valuing adding engineering activities
Respect and challenge suppliers (CS engineering teams)

18. Objectives of Lean Enterprise Principles*
Minimize waste Be responsive to change Right thing at the right place, at the right time, and in the right quantity Effective relationships (people and organizations) with the value stream Continuously improve processes and products Focus on quality from the beginning
* E. Murman, et al., Lean Enterprise Value: Insights from MIT’s Lean Aerospace Initiative. New York , NY: Palgrave, 2002

19. SoSE Process Strategies: Incorporation of Lean Enterprise Principles
SoSE guided by LAI Lean Enterprise 4 Grand Questions
Lean Enterprise 4 Grand Questions mapped to DoD SoSE case studies….
Lean Enterprise 4 Grand Questions mapped to SoSE core elements….
SoSE Core Element
Lean Enterprise Grand Questions
Stakeholder Considerations
Holistic Enterprise View
Q1: Understand Current
Q2: Future Possibilities
Q3: Strategies and Tactics for Future
Q4: Change Process
Translating Capability Objectives X
Understanding Systems and Relationships
Assessing Performance to Capability Objectives
Developing and Evolving an SoS Architecture
Monitoring and Assessing Changes
Addressing Requirements and Solution Options
Orchestrating Upgrades to SoS

20. SoSE Process Strategies: Engineering Cost Estimation
Calculations based on SoS characteristics/size and capability implementation approach using COSYSMO algorithm
SoSE effort
SoSE
Effort
Conversion to COSYSMO size units
CS 1 SoSE
contribution
effort
Equivalent
set of
“sea-level”
requirements
System
Capability
• • •
CS n SoSE
contribution
effort
Applies reuse factors, different cost factors for each engineering organization at each system level, and diseconomy of scale for SoS and CS-level requirements implemented in the same upgrade cycle….

21. SoSE Process Strategies: Artifacts
Artifacts can be characterized as key boundary objects between SoS and constituent system levels…

22. SoSE Process Strategies: Test and Evaluation
Key strategies
SoSE team/framework responsible for SoS-level testing
Build on CS-level testing
Use of test ranges and operational testing
Risk and evidence-based approach
Focus on
SoS capabilities
Network
Interoperability
Emergent behaviors
Performance assessment over time
Feedback process for fielded SoS

23. Case Study: Regional Area Crisis Response SoS (RACRS)23

24. Future SoSE Research Plans
Conduct deeper dives into
Lean lens analysis
Test and evaluation analysis
SoSE artifacts
SoS architecture evolution
RACRS model development and analysis techniques
SoSE cost model
Incorporate additional cost factors into COSYSMO to capture additional SoS characteristics and non-traditional SE effort
Evaluate impacts of insufficient SE at SoS level on rework
Break out SoSE effort
Planning
Implementation
SoS-level testing

25. Acknowledgements
DoD Director, Defense Research and Engineering (DDR&E)
Stevens-USC Systems Engineering Research Center (SERC) support
SoS case study work that has provided considerable engineering insights into SoSE
LAI for their research into lean enterprise concepts
Dr. Ricardo Valerdi’s COSYSMO cost model upon which the SoSE cost model is based

26. Questions Describe a key characteristic of an SoS.
List the key SoSE activities for the SoS agile rebaselining team.
Briefly describe 3 lean principles embedded in SoSE activities.

27. References
Dahmann, J. and K. Baldwin Understanding the current state of US defense systems of systems and the implications for systems engineering. Proceedings of the IEEE Systems Conference, April 7-10, in Montreal, Canada.
Department of Defense Systems engineering guide for system of systems, version 1.0.
Maier, M Architecting principles for systems-of-systems. Systems Engineering 1, no. 4:
Valerdi, R Constructive systems engineering cost model. PhD. Dissertation, University of Southern California.
Valerdi, R. and M. Wheaton ANSI/EIA 632 as a standardized WBS for COSYSMO, AIAA , Proceedings of the AIAA 5th Aviation, Technology, Integration, and Operations Conference, Arlington, Virginia.
Wang, G., R. Valerdi, A. Ankrum, C. Millar, and G. Roedler COSYSMO reuse extension, Proceedings of the 18th Annual International Symposium of INCOSE, The Netherlands.