1. University of Southern California Center for Systems and Software Engineering COSATMO/COSYSMO Workshop Jim Alstad, USC-CSSE Gan Wang, BAE Systems Garry J Roedler, Lockheed Martin April 30, 2014 04/29 1

2. University of Southern California Center for Systems and Software Engineering Workshop Agenda 04/292 Reminder: general directions for COSYSMO 3.0 development Delphi for SEFR and SEWR category parameter values –Overview of SEFR and SEWR –Instructions for Delphi questionnaires Discussion of extending COSYSMO to estimate development costs

3. University of Southern California Center for Systems and Software Engineering COSATMO/COSYSMO 3.0 Direction Several factors affecting the COSYSMO cost model have been shown to be valuable in increasing estimation accuracy: –Reuse (simple model—SEWR) –Reuse (with SEFR) –Requirements volatility (SERV) The rating scales for these could be integrated into a comprehensive COSYSMO model. –Which should provide more accurate estimates than any of these alone Add additional data points exhibiting a range of values for SEWR, SEFR, SERV Fit a COSYSMO III model to the overall dataset –Add variables and/or subset the data as needed 04/283

4. University of Southern California Center for Systems and Software Engineering COSATMO/COSYSMO Extension to Total Development Costs Explore a model for total development cost based primarily on the COSYSMO parameters (Roedler) –Can such a model be improved by dividing development cost into three parts: system engineering, hardware engineering, software engineering? (Alstad) 04/284

5. University of Southern California Center for Systems and Software Engineering Workshop Agenda 04/295 Reminder: general directions for COSYSMO 3.0 development Delphi for SEFR and SEWR category parameter values –Overview of SEFR and SEWR –Instructions for Delphi questionnaires Discussion of extending COSYSMO to estimate development costs

6. University of Southern California Center for Systems and Software Engineering What is this Delphi all about? A Generalized Reuse Framework implemented in COSYSMO –To assess not only the effort to leverage but also the effort to invest in a development project The goal is an industry-wide model for design sensitivity analysis and product line investment decisions –To bring back to organization to further tailor or use Invite your participation to calibrate the model coefficients, through a Delphi The Delphi process today… 6

7. University of Southern California Center for Systems and Software Engineering System Engineering For Reuse (SEFR) Producer’s View Production of reusable resources System Engineering With Reuse (SEWR) Consumer’s View Consumption of reusable resources Two Fundamental Reuse Processes 7

8. University of Southern California Center for Systems and Software Engineering CONOPS – Reuse in a Project 8 Total Project Effort = SEWR Effort + SEFR Effort

9. University of Southern California Center for Systems and Software Engineering Generalized Reuse Framework: System Engineering for Reuse System Engineering for Reuse produces artifacts intended for later reuse on projects. A completed SEFR artifact may (intentionally) not be completely developed, so that it will be in one of these SEFR states: –Conceptualized for Reuse (e.g., Concept of Operations document) –Designed for Reuse (e.g., component detailed design) –Constructed for Reuse (e.g., integrated component) –Validated for Reuse (e.g., validated component) 04/299

10. University of Southern California Center for Systems and Software Engineering Generalized Reuse Framework: System Engineering for Reuse 04/2910

11. University of Southern California Center for Systems and Software Engineering Generalized Reuse Framework: System Engineering with Reuse System Engineering with Reuse is project System Engineering, with reusable artifacts being brought into the product –A special case: zero reusable artifacts Each reusable artifact is included in one of these SEWR states of maturity: –New (i.e., not reused) –Re-implemented (through requirements & architecture) –Adapted (through detailed design) –Adopted (through implementation) –Managed (through system verification & validation) 04/2911

12. University of Southern California Center for Systems and Software Engineering Generalized Reuse Framework: System Engineering with Reuse 04/2912

13. University of Southern California Center for Systems and Software Engineering COSYSMO – Context COSYSMO –COnstructive SYStems Engineering Cost MOdel –Parametric Estimate of the Systems Engineering Effort –Covers full systems engineering lifecycle –Originally developed by Dr. Ricardo Valerdi and Dr. Barry Boehm at USC Inception of COSYSMO 1.0 –Valerdi, R., The Constructive Systems Engineering Cost Model (COSYSMO), PhD Dissertation, University of Southern California, May 2005. Introduced the Reuse Model Extension to COSYSMO 2.0 –Wang, G., Valerdi, R., Ankrum, A., Millar, C., and Roedler, G., “COSYSMO Reuse Extension,” Proceedings of the 18th INCOSE International Symposium, June 2008. –Fortune, J. Estimating Systems Engineering Reuse with the Constructive Systems Engineering Cost Model (COSYSMO 2.0). Ph.D. Dissertation. University of Southern California. December 2009 –Wang, G., Valerdi, R., Fortune, J., “Reuse in Systems Engineering,” IEEE System Journal, v4, No.3, 2010. Marching to COSYSMO 3.0 (work in progress…) –Fortune, J. and Valerdi, R., “Considerations for Successful Reuse in Systems Engineering,” AIAA Space 2008, San Diego, CA, September 2008. –Wang, G. and Rice, J., “Considerations for a Generalized Reuse Framework for System Development,” Proceedings of the 21st INCOSE International Symposium, June 2011. –Peña, M. Quantifying the Impact of Requirements Volatility on Systems Engineering Effort. Ph.D. Dissertation. University of Southern California. August 2012. –Fortune, J. and Valerdi, R., “A Framework for Systems Engineering Reuse,” Systems Engineering, 16(2), 2013. 13

14. University of Southern California Center for Systems and Software Engineering COSYSMO Overview (1/2) COSYSMO starts by computing the “size” of a system engineering project, in units of eReq (“equivalent nominal requirements”) These artifacts are considered in the size: system requirements, system interfaces, system-critical algorithms, and operational scenarios. Each artifact is evaluated as being easy, nominal, or difficult. Each artifact is looked up in this size table to get its number of eReq, and then these are summed to get the system size: 04/2914 Artifact TypeEasyNominalDifficult System Req’ts0.5 1.0 5.0 System Interfaces1.1 2.8 6.3 System Algs2.2 4.111.5 Op Scenarios6.214.430.0

15. University of Southern California Center for Systems and Software Engineering COSYSMO Overview (2/2) Size is raised to an exponent, representing diseconomy of scale, and then multiplied by factors for 14 effort multipliers and a calibration constant. This results in the following equation for a COSYSMO estimate of effort in person-months: 04/2915

16. University of Southern California Center for Systems and Software Engineering Generalized Reuse Framework: SEFR Model Equations A SEFR estimate adjusts each artifact’s size contribution by considering its SEFR state according to this table (from the first Delphi): 04/2916 SEFR State (Degree of Development)SEFR State Factor Conceptualized for Reuse36.98% Designed for Reuse58.02% Constructed for Reuse79.15% Validated for Reuse94.74%

17. University of Southern California Center for Systems and Software Engineering Generalized Reuse Framework: SEWR Model Equations A SEWR estimate adjusts each artifact’s size contribution by considering its SEWR state according to this table (from the first Delphi): 04/2917 SEWR State (Maturity)SEWR State Factor New100.00% Re-Implemented 66.73% Adapted 56.27% Adopted 38.80% Managed 21.70%

18. University of Southern California Center for Systems and Software Engineering The Exercise – Mapping Activities to Reuse Categories Format of Questionnaire Spreadsheet: –1 st tab: definition of reuse categories –2 nd tab: DWR questionnaire –3 rd tab: DFR questionnaire –4 th tab: definition of EIA-632 activitiesEIA-632 You do: –Fill out the cells intersecting EIA- 632 activities (rows) and the reuse categories (columns) –Enter the percentage you believe that particular activity applies or is relevant to a particular reuse category –Fill in a few words of your rationale for each selection 18 X

19. University of Southern California Center for Systems and Software Engineering Helpful Hints When evaluate the questionnaire, frame your thinking around the COSYSMO construct, i.e., a group of –Requirements –Interfaces –Algorithms –Scenarios Based on your overall professional experience, not a single or a particular program experience –Seek nominal behavior, biased toward desirable/more successful reuse situations The handouts have the definitions of reuse categories and EIA-632 activities For definition of general (engineering) terms, use INCOSE SE Handbook 19

20. University of Southern California Center for Systems and Software Engineering Questions and Comments on Delphi Contacts: Jim Alstad USC CSSE Los Angeles, CA jalstad@usc.edu Gan Wang BAE Systems Reston, VA gan.wang@baesystems.com Garry J. Roedler Lockheed Martin Philadelphia, PA garry.j.roedler@lmco.com 20

21. University of Southern California Center for Systems and Software Engineering Workshop Agenda 04/2921 Reminder: general directions for COSYSMO 3.0 development Delphi for SEFR and SEWR category parameter values –Overview of SEFR and SEWR –Instructions for Delphi questionnaires Discussion of extending COSYSMO to estimate development costs

22. University of Southern California Center for Systems and Software Engineering Extending COSYSMO To Estimate Development Cost The general goal: Use a model that is very similar to COSYSMO to estimate development cost There is some evidence that system engineering cost has some correlation with development cost Where to go from here? –Especially, who has data from actual projects that they are willing to contribute? 04/2922