1. University of Southern California Center for Systems and Software Engineering ICM Introduction Extracted from “Final Report: Integrating Systems and Software Engineering (IS&SE) Study” Barry Boehm and Jo Ann Lane University of Southern California Center for Systems and Software Engineering Arthur Pyster Stevens Institute of Technology December 31, 2007

2. University of Southern California Center for Systems and Software Engineering 12/31/2007 ©USC-CSSE 2 Outline DoD systems and software engineering trends and challenges State of systems and software engineering (S&SE) integration –Sources of gaps and challenges –Evaluation of current processes and standards Incremental Commitment Model (ICM) evaluation –ICM origin, principles, and organization –ICM evaluation Conclusions and recommendations References and acronyms

3. University of Southern California Center for Systems and Software Engineering 12/31/2007 ©USC-CSSE 3 ICM Practices and Assessment From the spiral model to the ICM –Principles and example Risk-driven incremental definition: ICM Stage I –Buying information to reduce risk Risk-driven incremental development: ICM Stage II –Achieving both rapid change and high assurance Multiple views of the ICM –Viewpoints and examples ICM Assessment

4. University of Southern California Center for Systems and Software Engineering 12/31/2007 ©USC-CSSE 4 From the Spiral Model to the ICM Need for intermediate milestones –Anchor Point Milestones (1996) Avoid stakeholder success model clashes –WinWin Spiral Model (1998) Avoid model misinterpretations –Essentials and variants (2000-2005) Clarify usage in DoD Instruction 5000.2 –Initial phased version (2005) Explain system of systems spiral usage to GAO –Underlying spiral principles (2006) Provide framework for human-systems integration –National Research Council report (2007)

5. University of Southern California Center for Systems and Software Engineering 12/31/2007 ©USC-CSSE 5 Process Model Principles 1.Commitment and accountability 2.Success-critical stakeholder satisficing 3.Incremental growth of system definition and stakeholder commitment 4, 5.Concurrent, iterative system definition and development cycles Cycles can be viewed as sequential concurrently- performed phases or spiral growth of system definition 6.Risk-based activity levels and anchor point commitment milestones

6. University of Southern California Center for Systems and Software Engineering 12/31/2007 ©USC-CSSE 6 Incremental Commitment in Gambling Total Commitment: Roulette –Put your chips on a number E.g., a value of a key performance parameter –Wait and see if you win or lose Incremental Commitment: Poker, Blackjack –Put some chips in –See your cards, some of others’ cards –Decide whether, how much to commit to proceed

7. University of Southern California Center for Systems and Software Engineering 12/31/2007 ©USC-CSSE 7 Scalable remotely controlled operations

8. University of Southern California Center for Systems and Software Engineering 12/31/2007 ©USC-CSSE 8 Total vs. Incremental Commitment – 4:1 RPV Total Commitment –Agent technology demo and PR: Can do 4:1 for $1B –Winning bidder: $800M; PDR in 120 days; 4:1 capability in 40 months –PDR: many outstanding risks, undefined interfaces –$800M, 40 months: “halfway” through integration and test –1:1 IOC after $3B, 80 months Incremental Commitment [number of competing teams] –$25M, 6 mo. to VCR [4]: may beat 1:2 with agent technology, but not 4:1 –$75M, 8 mo. to ACR [3]: agent technology may do 1:1; some risks –$225M, 10 mo. to DCR [2]: validated architecture, high-risk elements –$675M, 18 mo. to IOC [1]: viable 1:1 capability –1:1 IOC after $1B, 42 months

9. University of Southern California Center for Systems and Software Engineering 12/31/2007 ©USC-CSSE 9 The Cone of Uncertainty: Usual result of total commitment ^ Inadequate PDR Better to buy information to reduce risk

10. University of Southern California Center for Systems and Software Engineering 12/31/2007 ©USC-CSSE 10 ICM Principles and Approach From the spiral model to the ICM –Principles and example Risk-driven incremental definition: ICM Stage I –Buying information to reduce risk Risk-driven incremental development: ICM Stage II –Achieving both rapid change and high assurance Multiple views of the ICM –Viewpoints and examples

11. University of Southern California Center for Systems and Software Engineering 12/31/2007 ©USC-CSSE 11 The Incremental Commitment Life Cycle Process: Overview Stage I: DefinitionStage II: Development and Operations Anchor Point Milestones Synchronize, stabilize concurrency via FRs Risk patterns determine life cycle process

12. University of Southern California Center for Systems and Software Engineering 12/31/2007 ©USC-CSSE 12 Anchor Point Feasibility Rationales Evidence provided by developer and validated by independent experts that: If the system is built to the specified architecture, it will –Satisfy the requirements: capability, interfaces, level of service, and evolution –Support the operational concept –Be buildable within the budgets and schedules in the plan –Generate a viable return on investment –Generate satisfactory outcomes for all of the success-critical stakeholders All major risks resolved or covered by risk management plans Serves as basis for stakeholders’ commitment to proceed

13. University of Southern California Center for Systems and Software Engineering 12/31/2007 ©USC-CSSE 13 There is Another Cone of Uncertainty: Shorter increments are better Uncertainties in competition, technology, organizations, mission priorities

14. University of Southern California Center for Systems and Software Engineering 12/31/2007 ©USC-CSSE 14 The Incremental Commitment Life Cycle Process: Overview Stage I: DefinitionStage II: Development and Operations Anchor Point Milestones Concurrently engr. OpCon, rqts, arch, plans, prototypes Concurrently engr. Incr.N (ops), N+1 (devel), N+2 (arch)

15. University of Southern California Center for Systems and Software Engineering 12/31/2007 ©USC-CSSE 15 ICM Stage II: Increment View

16. University of Southern California Center for Systems and Software Engineering 12/31/2007 ©USC-CSSE 16 ICM Stage II: Increment View A radical idea? No; a commercial best practice and part of DoDI 5000.2

17. University of Southern California Center for Systems and Software Engineering 12/31/2007 ©USC-CSSE 17 ICM Principles and Approach From the spiral model to the ICM –Principles and example Risk-driven incremental definition: ICM Stage I –Buying information to reduce risk Risk-driven incremental development: ICM Stage II –Achieving both rapid change and high assurance Multiple views of the ICM –Viewpoints and examples

18. University of Southern California Center for Systems and Software Engineering 12/31/2007 ©USC-CSSE 18 V C R D C R I O C O C R A C R C C D MBASE-RUP/ICM Anchor Points Enable Concurrent Engineering

19. University of Southern California Center for Systems and Software Engineering 12/31/2007 ©USC-CSSE 19 ICM HSI Levels of Activity for Complex Systems

20. University of Southern California Center for Systems and Software Engineering 12/31/2007 ©USC-CSSE 20 Different Risk Patterns Yield Different Processes

21. University of Southern California Center for Systems and Software Engineering 12/31/2007 ©USC-CSSE 21 Special CaseExampleSize, Complexit y Change Rate % /Month CriticalityNDI SupportOrg, Personnel Capability Key Stage I Activities : Incremental Definition Key Stage II Activities: Incremental Development, Operations Time per Build; per Increment 1. Use NDISmall AccountingCompleteAcquire NDIUse NDI 2. AgileE-servicesLow1 – 30Low-MedGood; in place Agile-ready Med-high Skip Valuation, Architecting phasesScrum plus agile methods of choice<= 1 day; 2-6 weeks 3. Scrum of Scrums Business data processing Med1 – 10Med-HighGood; most in place Agile-ready Med-high Combine Valuation, Architecting phases. Complete NDI preparation Architecture-based Scrum of Scrums 2-4 weeks; 2-6 months 4. SW embedded HW component Multisensor control device Low0.3 – 1Med-Very High Good; In place Experienced; med-high Concurrent HW/SW engineering. CDR-level ICM DCR IOC Development, LRIP, FRP. Concurrent Version N+1 engineering SW: 1-5 days; Market-driven 5. Indivisible IOCComplete vehicle platform Med – High 0.3 – 1High- Very High Some in placeExperienced; med-high Determine minimum-IOC likely, conservative cost. Add deferrable SW features as risk reserve Drop deferrable features to meet conservative cost. Strong award fee for features not dropped SW: 2-6 weeks; Platform: 6-18 months 6. NDI- IntensiveSupply Chain Management Med – High 0.3 – 3Med- Very High NDI-driven architecture NDI-experienced; Med-high Thorough NDI-suite life cycle cost- benefit analysis, selection, concurrent requirements/ architecture definition Pro-active NDI evolution influencing, NDI upgrade synchronization SW: 1-4 weeks; System: 6-18 months 7. Hybrid agile / plan-driven system C4ISRMed – Very High Mixed parts: 1 – 10 Mixed parts; Med-Very High Mixed parts Full ICM; encapsulated agile in high change, low-medium criticality parts (Often HMI, external interfaces) Full ICM,three-team incremental development, concurrent V&V, next- increment rebaselining 1-2 months; 9-18 months 8. Multi-owner system of systems Net-centric military operations Very High Mixed parts: 1 – 10 Very High Many NDIs; some in place Related experience, med- high Full ICM; extensive multi-owner team building, negotiation Full ICM; large ongoing system/software engineering effort 2-4 months; 18- 24 months 9. Family of systems Medical Device Product Line Med – Very High 1 – 3Med – Very High Some in placeRelated experience, med – high Full ICM; Full stakeholder participation in product line scoping. Strong business case Full ICM. Extra resources for first system, version control, multi- stakeholder support 1-2 months; 9- 18 months Common Risk-Driven Special Cases of the Incremental Commitment Model (ICM) C4ISR: Command, Control, Computing, Communications, Intelligence, Surveillance, Reconnaissance. CDR: Critical Design Review. DCR: Development Commitment Review. FRP: Full-Rate Production. HMI: Human-Machine Interface. HW: Hard ware. IOC: Initial Operational Capability. LRIP: Low-Rate Initial Production. NDI: Non-Development Item. SW: Software

22. University of Southern California Center for Systems and Software Engineering 12/31/2007 ©USC-CSSE 22 References - I Beck, K., Extreme Programming Explained, Addison Wesley, 1999. Boehm, B., “Some Future Trends and Implications for Systems and Software Engineering Processes”, Systems Engineering 9(1), pp. 1-19, 2006. Boehm, B., Brown, W., Basili, V., and Turner, R., “Spiral Acquisition of Software-Intensive Systems of Systems, CrossTalk, Vol. 17, No. 5, pp. 4-9, 2004. Boehm, B. and Lane J., "21st Century Processes for Acquiring 21st Century Software-Intensive Systems of Systems." CrossTalk: Vol. 19, No. 5, pp.4-9, 2006. Boehm, B., and Lane, J., “Using the ICM to Integrate System Acquisition, Systems Engineering, and Software Engineering,” CrossTalk, October 2007, pp. 4-9. Boehm, B., “Future Challenges and Rewards for Softwarre Engineers,” DoD Software Tech News, October 2007, pp. 6-12. Boehm, B. et al., Software Cost Estimation with COCOMO II, Prentice Hall, 2000. Boehm, B., Software Engineering Economics, Prentice Hall, 2000. Carlock, P. and Fenton, R., "System of Systems (SoS) Enterprise Systems for Information-Intensive Organizations," Systems Engineering, Vol. 4, No. 4, pp. 242-26, 2001. Carlock, P., and J. Lane, “System of Systems Enterprise Systems Engineering, the Enterprise Architecture Management Framework, and System of Systems Cost Estimation”, 21st International Forum on COCOMO and Systems/Software Cost Modeling, 2006. Checkland, P., Systems Thinking, Systems Practice, Wiley, 1980 (2 nd ed., 1999). Department of Defense (DoD), Defense Acquisition Guidebook, version 1.6, http://akss.dau.mil/dag/, 2006.http://akss.dau.mil/dag/ Hall, E.T., Beyond Culture, Anchor Books/Doubleday, 1976. Highsmith, J., Adaptive Software Development, Dorset House, 2000. Jensen, R. “An Improved Macrolevel Software Development Resource Estimation Model,” Proceedings, ISPA 5, April 1983, pp. 88-92.

23. University of Southern California Center for Systems and Software Engineering 12/31/2007 ©USC-CSSE 23 References -II Krygiel, A. (1999); Behind the Wizard’s Curtain; CCRP Publication Series, July, 1999, p. 33 Lane, J. and Boehm, B., "System of Systems Cost Estimation: Analysis of Lead System Integrator Engineering Activities", Information Resources Management Journal, Vol. 20, No. 2, pp. 23-32, 2007. Lane, J. and Valerdi, R., “Synthesizing SoS Concepts for Use in Cost Estimation”, Proceedings of IEEE Systems, Man, and Cybernetics Conference, 2005. Lientz, B., and Swanson, E.B., Software Maintenance Management, Addison Wesley, 1980. Madachy, R., Boehm, B., Lane, J., "Assessing Hybrid Incremental Processes for SISOS Development", USC CSSE Technical Report USC-CSSE-2006-623, 2006. Maier, M., “System and Software Architecture Reconciliation,” Systems Engineering 9 (2), 2006, pp. 146- 159. Northrop, L., et al., Ultra-Large-Scale Systems: The Software Challenge of the Future, Software Engineering Institute, 2006. Pew, R. W., and Mavor, A. S., Human-System Integration in the System Development Process: A New Look, National Academy Press, 2007. Putnam, L., “A General Empirical Solution to the Macro Software Sizing and Estimating Problem,” IEEE Trans SW Engr., July 1978, pp. 345-361. Rechtin, E. Systems Architecting, Prentice Hall, 1991. Schroeder, T., “Integrating Systems and Software Engineering: Observations in Practice,” OSD/USC Integrating Systems and Software Engineering Workshop, http://csse.usc.edu/events/2007/CIIForum/pages/program.html, October 2007. http://csse.usc.edu/events/2007/CIIForum/pages/program.html

24. University of Southern California Center for Systems and Software Engineering 12/31/2007 ©USC-CSSE 24 SoSE-Related References Carlock, P.G., and R.E. Fenton, "System of Systems (SoS) Enterprise Systems for Information-Intensive Organizations," Systems Engineering, Vol. 4, No. 4, pp. 242-261, 2001 Department of Defense (DoD), Defense Acquisition Guidebook, version 1.6, http://akss.dau.mil/dag/, 2006.http://akss.dau.mil/dag/ Department of Defense (DoD), System of Systems Engineering Guide: Considerations for Systems Engineering in a System of Systems Environment, draft version 0.9, 2006. DiMario, Mike (2006); “System of Systems Characteristics and Interoperability in Joint Command Control”, Proceedings of the 2nd Annual System of Systems Engineering Conference Electronic Industries Alliance (1999); EIA Standard 632: Processes for Engineering a System Finley, James (2006); “Keynote Address”, Proceedings of the 2nd Annual System of Systems Engineering Conference Garber, Vitalij (2006); “Keynote Presentation”, Proceedings of the 2nd Annual System of Systems Engineering Conference INCOSE (2006); Systems Engineering Handbook, Version 3, INCOSE-TP-2003-002-03 Kuras, M. L., White, B. E., Engineering Enterprises Using Complex-System Engineering, INCOSE Symposium 2005. Krygiel, A. (1999); Behind the Wizard’s Curtain; CCRP Publication Series, July, 1999, p. 33 Maier, M. (1998); “Architecting Principles for Systems-of-Systems”; Systems Engineering, Vol. 1, No. 4 (pp 267-284) Meilich, Abe (2006); “System of Systems Engineering (SoSE) and Architecture Challenges in a Net Centric Environment”, Proceedings of the 2nd Annual System of Systems Engineering Conference Pair, Major General Carlos (2006); “Keynote Presentation”, Proceedings of the 2nd Annual System of Systems Engineering Conference Proceedings of AFOSR SoSE Workshop, Sponsored by Purdue University, 17-18 May 2006 Proceedings of the 2nd Annual System of Systems Engineering Conference, Sponsored by System of Systems Engineering Center of Excellence (SoSECE), http://www.sosece.org/, 2006.http://www.sosece.org/ Proceedings of Society for Design and Process Science 9 th World Conference on Integrated Design and Process Technology, San Diego, CA, 25-30 June 2006 Siel, Carl (2006); “Keynote Presentation”, Proceedings of the 2nd Annual System of Systems Engineering Conference United States Air Force Scientific Advisory Board (2005); Report on System-of-Systems Engineering for Air Force Capability Development; Public Release SAB-TR-05-04

25. University of Southern California Center for Systems and Software Engineering 12/31/2007 ©USC-CSSE 25 List of Acronyms ACRArchitecting Commitment Review B/LBaselined CCDCore Capability Drive-Through COTSCommercial Off-the-Shelf DCRDevelopment Commitment Review DIDevelopment Increment DOTMLPFDoctrine, Organization, Training, Materiel, Leadership, Personnel, Facilities ECRExploration Commitment Review FMEAFailure Modes and Effects Analysis GUIGraphical User Interface HSIHuman-System Interface ICMIncremental Commitment Model IOCInitial Operational Capability IRRInception Readiness Review

26. University of Southern California Center for Systems and Software Engineering 12/31/2007 ©USC-CSSE 26 List of Acronyms (continued) LCALife Cycle Architecture LCOLife Cycle Objectives OCOperational Capability OCROperations Commitment Review OO&DObserve, Orient and Decide OODAObserve, Orient, Decide, Act O&MOperations and Maintenance PRRProduct Release Review RACRSRegional Area Crisis Response System SoSSystem of Systems SoSESystem of Systems Engineering TSETraditional Systems Engineering VCRValidation Commitment Review V&VVerification and Validation