1. Expert COSYSMO Update Raymond Madachy rjmadach@nps.edu USC-CSSE Annual Research Review March 17, 2009

2. 2 Expert COSYSMO Introduction An expert system tool for systems engineering risk management based on the Constructive Systems Engineering Cost Model (COSYSMO) [Valerdi 2005] –Automatically identifies project risks in conjunction with cost estimation similar to Expert COCOMO [Madachy 1997] and provides related advice –Supports project planning by identifying, categorizing and quantifying system-level risks –Supports project execution with automated risk mitigation advice for management consideration Risk situations are characterized by combinations of cost driver values indicating increased effort with a potential for more problems –Includes 98 risk conditions Simultaneously calculates cost to enable tradeoffs with risk https://diana.nps.edu/MSAcq/tools/ExpertCOSYSMO.php http://csse.usc.edu/tools/ExpertCOSYSMO.php

3. 3 Method Analyzes patterns of cost driver ratings submitted for a COSYSMO cost estimate against pre-determined risk rules –Identifies individual risks that an experienced systems engineering manager might recognize but often fails to take into account –Helps users determine and rank sources of project risk. With these risks, mitigation plans are created based on the relative risk severities and provided advice

4. 4 Method (cont.) COSYSMO cost factor combinations used as abstractions for formulating risk heuristics –E.g. if Architecture Understanding = Very Low and Level of Service Requirements = Very High, then there is a high risk Since systems with high service requirements are more difficult to implement especially when the architecture is not well understood Elicitation of knowledge from systems engineering domain experts in CSSE-sponsored workshops –Survey used to identify and quantify risks Devised knowledge representation scheme and risk quantification algorithm All risk rules are fired when the effort multipliers of both cost factors involved are > 1.0 –Can be more granular for gradations of risk Recent extension for risk mitigation advice

5. 5 Risk Conditions

6. 6 Risk Taxonomy and Weighting ProjectRiskrisklevel effort multiplier product,    ij i categoryrisks j categories ij, * ## 11 where risk level =1moderate 2high 4very high effort multiplier product= (driver #1 effort multiplier) * (driver #2 effort multiplier)... * (driver #n effort multiplier). Project Risk Product riskProcess riskPersonnel riskPlatform risk

7. 7 Expert COSYSMO Inputs

8. 8 Expert COSYSMO Outputs

9. New Risk Mitigation Advice 9 All risks triggered when both related effort multipliers are > 1.0 See handout for review

10. Size Risk Elaboration Need to establish size range thresholds for risk rules COSYSMO size distribution: Min = 82, Max = 17,763 equivalent requirements Proposed ranges –Small: < 5,000 equivalent requirements –Medium: between 5,000 and 15,000 equivalent requirements –Large: > 15,000 equivalent requirements See handout on size related risks 10

11. Finer Assignment of Risk Levels Different severity risks would entail contours of different strength Some risk combinations may require re-interpretation of the nominal boundaries

12. 12 Current and Future Work Scaling the risk summary outputs for each category and defining ranges for low, medium and high risks Create more granular risk quantification rules Consider 3-way risk interactions Add rules to detect COSYSMO input anomalies Systems engineering risk data from industrial projects will be analyzed to enhance and refine the technique –Perform statistical testing Domain experts from industry and government will continue to provide feedback and clarification –Supporting surveys and workshops will be continued Integrating alternate risk uncertainty approaches into a more complete risk management framework –Recently added Monte-Carlo analysis

13. Tasks So Far COSYSMO workshop experts have identified and prioritized risks, and provided advice Researchers devised the risk taxonomy and its weighting scheme Today We invite your comments on all Help us review and complete the advice 13

14. Assignments Dan/Beth pp. 1-2 Jared/Stan p. 3 Mauricio/Garry p. 4 Miles/Ricardo p. 5 and size risks 14

15. 15 References R. Madachy, Heuristic Risk Assessment Using Cost Factors, IEEE Software, May 1997 Valerdi R., The Constructive Systems Engineering Cost Model (COSYSMO), PhD Dissertation, University of Southern California, Los Angeles, CA, May 2005 http://csse.usc.edu/tools/ExpertCOSYSMO.php