1. University of Southern California Center for Systems and Software Engineering COSYSMO 3.0: Workshop Results Annual Research Review March 17, 2016 03/16 1 Jim AlstadUSC Center for Systems and Software Engineering Jim Alstad*310/344-0894jalstad@usc.edu

2. University of Southern California Center for Systems and Software Engineering Workshop Agenda 03/162 Tentative agenda: 13:00 Attendee introductions 13:15 Detailed introduction to the model and to workshop techniques 13:30 Delphi round 1 15:00 Break 15:30 Delphi round 2 16:30 Closing discussions 17:00 End

3. University of Southern California Center for Systems and Software Engineering Workshop Materials 03/163 Main presentation (from this morning) Workshop presentation (this presentation) Model document Delphi voting form

4. University of Southern California Center for Systems and Software Engineering Wideband Delphi Procedure 03/164 Ballots available via: –E-mail (Excel) –Memory stick (Excel) –Paper Round 1 –Moderator goes over the parameters in the model –Each voter fills in a ballot with her opinion as to the correct value of the parameter –Ballots are anonymous –No discussion –When complete, ballots are turned in to moderator Later round –Moderator assembles results of voting and presents to group –Moderator leads group discussion of results –Voters fill in a new ballot with possibly revised parameter values

5. University of Southern California Center for Systems and Software Engineering Context for this Delphi (1/2) 03/165 Productivity range (“EMR”): –Ratio of the numerically highest value for a cost driver to its numerically lowest value –Is a measure of the impact of the cost driver –Recommended voter concept: “Considering the project with the best rating on this parameter to the project with the worst rating, what is the productivity ratio due to this parameter?” Some commentators have suggested that the EMRs as a group are too large –I.e., the product of all EMRs is too large –Consequence can be that, for example, going from Nominal to High (one step) causes too big a change in the estimate

6. University of Southern California Center for Systems and Software Engineering Context for this Delphi (2/2) 03/166 One commentator suggests that a low capability team (Personnel/Team Capability) may also have low CONOPS and Requirements Understanding and low Architecture Understanding, leading to an unwarranted high value for the product of these. –Currently, product of EMRs = 19.66

7. University of Southern California Center for Systems and Software Engineering Discussion Topix 03/167 Jim/Winsor: With/without Interoperability –Jim: Nominal should be the same Resolved: Jim’s misunderstanding: two “separate” tables –Winsor: No further discussion needed Jim: Somebody lowered all Cost Drivers –Resolve: We’ve discussed this. Our opinions will go into Round 2 Winsor/Hunter: Scale factors: Lower base, increase RV –Resolve: Allow scale factors to be +/-, including RV. Allow for asymmetry. Marilee: # Recursive Levels –Not for group resolution Jim: Statement for RV –Resolve: Use perdcentages