1. University of Southern California Center for Systems and Software Engineering ©USC-CSSE1 Ray Madachy, Ricardo Valerdi USC Center for Systems and Software Engineering MIT Lean Aerospace Initiative madachy@usc.edu, rvalerdi@mit.edu 22nd International Forum on COCOMO and Systems/Software Cost Modeling November 1, 2007 Expert COSYSMO

2. University of Southern California Center for Systems and Software Engineering ©USC-CSSE2 Introduction An expert system tool for systems engineering risk assessment based on the Constructive Systems Engineering Cost Model (COSYSMO) [Valerdi 2005] –Automatically identifies project risks in conjunction with cost estimation similar to Expert COCOMO [Madachy 1997] –Supports project planning by identifying, categorizing, quantifying, and prioritizing system-level risks –Includes 98 risk conditions Risk situations are characterized by combinations of cost driver values indicating increased effort with a potential for more problems Simultaneously calculates cost to enable tradeoffs with risk

3. University of Southern California Center for Systems and Software Engineering ©USC-CSSE3 Method Analyzes patterns of cost driver ratings submitted for a COSYSMO cost estimate against pre-determined risk rules –Identifies individual risks that an experienced systems engineering manager might recognize but often fails to take into account –Helps users determine and rank sources of project risk. With these risks, mitigation plans can be created based on the relative risk severities and provided advice

4. University of Southern California Center for Systems and Software Engineering ©USC-CSSE4 Method (cont.) COSYSMO cost factor combinations used as abstractions for formulating risk heuristics –E.g. if Architecture Understanding = Very Low and Level of Service Requirements = Very High, then there is a high risk Since systems with high service requirements are more difficult to implement especially when the architecture is not well understood Elicitation of knowledge from systems engineering domain experts in CSSE-sponsored workshops –Survey used to identify and quantify risks Devised knowledge representation scheme and risk quantification algorithm

5. University of Southern California Center for Systems and Software Engineering ©USC-CSSE5 Risk Conditions

6. University of Southern California Center for Systems and Software Engineering ©USC-CSSE6 Risk Taxonomy and Weighting ProjectRiskrisklevel effort multiplier product,    ij i categoryrisks j categories ij, * ## 11 where risk level =1moderate 2high 4very high effort multiplier product= (driver #1 effort multiplier) * (driver #2 effort multiplier)... * (driver #n effort multiplier). Project Risk Product riskProcess riskPersonnel riskPlatform risk

7. University of Southern California Center for Systems and Software Engineering ©USC-CSSE7 Next: Finer Assignment of Risk Levels ATTRIBUTE 1 very low extra high very low very high high moderate ATTRIBUTE 2 very high discretized into ATTRIBUTE 1 VERY LOW LOW NOMINAL HIGHVERY HIGHEXTRA HIGH VERY LOWMODERATEHIGHVERY HIGH LOWMODERATEHIGH ATTRIBUTE 2 NOMINALMODERATE HIGH VERY HIGH increasing risk

8. University of Southern California Center for Systems and Software Engineering ©USC-CSSE8 Expert COSYSMO Inputs

9. University of Southern California Center for Systems and Software Engineering ©USC-CSSE9 Expert COSYSMO Outputs

10. University of Southern California Center for Systems and Software Engineering ©USC-CSSE10 Current and Future Work Currently scaling the risk summary outputs for each category and defining ranges for low, medium and high risks Create more granular risk quantification rules Generate expert risk mitigation advice for each risk condition, and provide that automated guidance to users to help develop their own mitigation actions Add rules to detect COSYSMO input anomalies Systems engineering risk data from industrial projects will be analyzed to enhance and refine the technique –Perform statistical testing Domain experts from industry and government will continue to provide feedback and clarification –Supporting surveys and workshops will be continued Exploration of alternate risk and uncertainty approaches to integrate multiple risk management viewpoints into a more complete risk management framework

11. University of Southern California Center for Systems and Software Engineering ©USC-CSSE11 References R. Madachy, Heuristic Risk Assessment Using Cost Factors, IEEE Software, May 1997 Valerdi R., The Constructive Systems Engineering Cost Model (COSYSMO), PhD Dissertation, University of Southern California, Los Angeles, CA, May 2005 http://csse.usc.edu/tools/expert_cosysmo.php