1. University of Southern California Center for Systems and Software Engineering 10/27/2008©USC-CSSE1 Predicting Understandability of a Software Project Using COCOMO II Model Drivers Ali Afzal Malik Barry Boehm A. Winsor Brown {alimalik, boehm, awbrown} @usc.edu 23 rd International Forum on COCOMO and Systems/Software Cost Modeling

2. University of Southern California Center for Systems and Software Engineering 10/27/2008©USC-CSSE2 Outline Introduction Motivation & Related Work Methodology Results Future Work References

3. University of Southern California Center for Systems and Software Engineering 10/27/2008©USC-CSSE3 Introduction Understandability –“Degree of clarity of the purpose and requirements of a software system to the developers of that system at the end of the Inception phase” Basic idea –Quantification enables prediction –Reuse inputs of software cost estimation Empirical Study –Projects

4. University of Southern California Center for Systems and Software Engineering 10/27/2008©USC-CSSE4 RUP Hump Chart (Kruchten, 2003)

5. University of Southern California Center for Systems and Software Engineering 10/27/2008©USC-CSSE5 Empirical Study SE I (Fall) and SE II (Spring) 2004 – 2007 24 real-client, MS-student, team projects (SE I 2008, SE II 2008) Process: MBASE/RUP (Boehm et al. 2005, Kruchten 2003) Projects –Development-intensive –Used COCOMO II

6. University of Southern California Center for Systems and Software Engineering 10/27/2008©USC-CSSE6 S#YearProjectType 12004Bibliographies on Chinese Religions in Western LanguagesWeb-based database 22004Data Mining of Digital Library Usage DataData mining 32004Data Mining from Report FilesData mining 42005Data Mining PubMed ResultsData mining 52005EBay Notification SystemStand-alone application 62005Rule-based EditorGUI 72005CodeCount™ Product Line with XML and C++Code Counter Tool 82006California Science Center Newsletter SystemWeb-based database 92006California Science Center Event RSVP SystemWeb-based database 102006USC Diploma Order/ Tracking Database SystemWeb-based database 112006USC Civic and Community Relations web applicationWeb-based database 122006Student's academic progress web applicationWeb-based database 132006New Economics for Woman (NEW)Web-based database 142006Web Portal for USC Electronic ResourcesWeb-based GUI 152006Early Medieval East Asian TombsWeb-based database 162006USC CONIPMOCost model 172006An Eclipse Plug-in for Use Case AuthoringStand-alone application 182007USC COINCOMOCost model 192007BTI Appraisal ProjectsStand-alone database 202007LAMAS Customer Service ApplicationWeb-based database 212007BID review SystemStand-alone database 222007Proctor and Test Site Tracking SystemWeb-based database 232007E-Mentoring programWeb-based application 242007Los Angeles County Generation Web InitiativeWeb-based database

7. University of Southern California Center for Systems and Software Engineering 10/27/2008©USC-CSSE7 Motivation & Related Work Some important considerations –18% of software project failures due to unclear objectives and incomplete R&S (Standish Group 1995) –Escalation in cost of fixing requirements defects: rapid for large and considerable for smaller projects (Boehm 1981, Boehm and Turner 2004) –Requirement changes have significant impact on project’s budget and schedule (Zowghi and Nurmuliani 2002)

8. University of Southern California Center for Systems and Software Engineering 10/27/2008©USC-CSSE8 Motivation & Related Work (2) An objective mechanism to predict understandability enables –Minimization of resource wastage due to rework –Answering “How much RE is enough?” Related previous work –“Expert COCOMO” (Madachy 1997) Uses COCOMO II cost factors to quantify risk

9. University of Southern California Center for Systems and Software Engineering 10/27/2008©USC-CSSE9 Methodology Identified 8 relevant COCOMO II model drivers S#Model DriverDescription 1PRECProduct precedentedness 2RESLArchitecture/Risk resolution 3CPLXProduct complexity 4ACAPAnalyst capability 5PCAPProgrammer capability 6APEXApplications experience 7PLEXPlatform experience 8LTEXLanguage and tool experience

10. University of Southern California Center for Systems and Software Engineering 10/27/2008©USC-CSSE10 Methodology (2) Weighted-sum formula –UNDR – understandability –MD i – i th Model Driver’s value –w i – weight of MD i –n i – nature of MD i ; Є {-1, +1} -1 for CPLX; +1 for the rest

11. University of Southern California Center for Systems and Software Engineering 10/27/2008©USC-CSSE11 Methodology (3) Model driver rating scale Driver RatingRating SymbolNumerical Value Very LowVL1 LowL2 NominalN3 HighH4 Very HighVH5 Extra HighXH6

12. University of Southern California Center for Systems and Software Engineering 10/27/2008©USC-CSSE12 Methodology (4) Voting for model driver weights –22 students from SE II class –Rating scale 1 (least important) – 5 (most important) iMD i wiwi 1PREC3.64 2RESL3.23 3CPLX3.64 4ACAP3.55 5PCAP3.95 6APEX3.64 7PLEX3.32 8LTEX3.36

13. University of Southern California Center for Systems and Software Engineering 10/27/2008©USC-CSSE13 Methodology (5) Determine the lowest and highest numerical values of understandability

14. University of Southern California Center for Systems and Software Engineering 10/27/2008©USC-CSSE14 Methodology (6) min() and max() min (MD i ) { if (n i = = +1) return minimum numerical value of MD i else return maximum numerical value of MD i } max (MD i ) { if (n i = = +1) return maximum numerical value of MD i else return minimum numerical value of MD i }

15. University of Southern California Center for Systems and Software Engineering 10/27/2008©USC-CSSE15 Methodology (7) iMD i nini wiwi min(MD i )max(MD i ) 1PREC+13.6416 2RESL+13.2316 3CPLX3.6461 4ACAP+13.5515 5PCAP+13.9515 6APEX+13.6415 7PLEX+13.3215 8LTEX+13.3615

16. University of Southern California Center for Systems and Software Engineering 10/27/2008©USC-CSSE16 Methodology (8) Project categories using UNDR ranges –Vaguely-understood applications (VUA) –Intermediate-clarity applications (ICA) –Well-understood applications (WUA) 2.8644.1285.38126.64 VUAICAWUA UNDR Low UNDR High

17. University of Southern California Center for Systems and Software Engineering 10/27/2008©USC-CSSE17 Methodology (9) Ranges defining VUA, ICA, and WUA groups are adjustable –End-points of spectrum depend on weights –Range sizes can be non-uniform

18. University of Southern California Center for Systems and Software Engineering 10/27/2008©USC-CSSE18 Results S#UNDR ValuePredicted UNDR GroupActual UNDR GroupMatches reality? 164.05ICA Yes 252.23ICA Yes 373.41ICA Yes 480.82ICA Yes 549.59ICAVUANo 676.59ICA Yes 770.09ICA Yes 859.50ICA Yes 967.09ICA Yes 1063.45ICA Yes 1162.95ICA Yes 1273.64ICA Yes 1359.59ICA Yes 1431.77VUA Yes 1569.68ICA Yes 1684.59WUA Yes 1724.50VUA Yes 1876.91ICA Yes 1962.82ICAVUANo 2077.18ICA Yes 2156.18ICA Yes 2284.18ICAWUANo 2373.41ICA Yes 2466.32ICAWUANo

19. University of Southern California Center for Systems and Software Engineering 10/27/2008©USC-CSSE19 Results (2) Prediction accuracy –ICA: 100% –VUA: 50% (2 out of 4) –WUA: 33% (1 out of 3) –Overall: 83% (20 out of 24) Possible reasons for discrepancies –Inappropriate COCOMO II model drivers –Voters are different from developers Projects # 5 & 22

20. University of Southern California Center for Systems and Software Engineering 10/27/2008©USC-CSSE20 Future Work Weights for commercial projects using techniques such as Wideband Delphi (Boehm 1981) Investigate other widely-used models e.g. SLIM (Putnam 1978) and PRICE-S (Freiman and Park 1979) Analyze understandability’s contribution towards degree of requirements elaboration (Malik and Boehm 2008)

21. University of Southern California Center for Systems and Software Engineering 10/27/2008©USC-CSSE21 References Books –Boehm, B., Software Engineering Economics, Prentice-Hall, 1981. –Boehm, B., Abts, C., Brown, A., Chulani, S., Clark, B, Horowitz, E., Madachy, R., Reifer, D., and Steece, B., Software Cost Estimation with COCOMO II, Prentice Hall, 2000. –Boehm, B. and Turner, R., Balancing Agility and Discipline: A Guide for the Perplexed, Addison- Wesley, 2004. –Kruchten, P., The Rational Unified Process: An Introduction, Addison-Wesley, 2003. Conference papers –Boehm, B., “Anchoring the Software Process”, IEEE Software 13(4), 1996, pages 73-82. –Freiman, F.R. and Park, R. E., “PRICE Software Model–Version 3: An Overview”, Proc. IEEE-PINY Workshop on Quantitative Software Models, 1979, pages 32-41. –Madachy, R., “Heuristic Risk Assessment Using Cost Factors”, IEEE Software 4 (3), 1997, pages 51-59. –Malik, A. A. and Boehm, B., “An Empirical Study of Requirements Elaboration”, The 22nd Brazilian Symposium on Software Engineering (SBES’08), 2008. –Putnam, L. H., “A General Empirical Solution to the Macro Software Sizing and Estimating Problem”, IEEE Trans. Software Engr., 1978, pages 345–361. –Zowghi, D. and Nurmuliani, N., “A Study of the Impact of Requirements Volatility on Software Project Performance”, Proceedings of the Ninth Asia-Pacific Software Engineering Conference, 2002, pages 3-11. Miscellaneous –Boehm, B., Klappholz, D., Colbert, E., et al., “Guidelines for Lean Model-Based (System) Architecting and Software Engineering (LeanMBASE)”, Center for Software Engineering, University of Southern California, 2005. –SE I (2008). Links to websites of all past semesters of Software Engineering I (CSCI 577A) course at USC, http://sunset.usc.edu/csse/courseroot/course_list.html#577a –SE II (2008). Links to websites of all past semesters of Software Engineering II (CSCI 577B) course at USC, http://sunset.usc.edu/csse/courseroot/course_list.html#577b –Standish Group (1995). “CHAOS”, http://www.standishgroup.com