1. University of Southern California Center for Systems and Software Engineering © 2009, USC-CSSE 1 An Analysis of Changes in Productivity and COCOMO Cost Drivers over the Years Vu Nguyen, Barry Boehm Center for Systems and Software Engineering (CSSE) 24 th Int’l COCOMO Forum Nov 3 rd, 2009

2. University of Southern California Center for Systems and Software Engineering © 2009, USC-CSSE 2 Outline Objectives and Background Productivity Changes Discussions and Conclusions Cost Driver Changes

3. University of Southern California Center for Systems and Software Engineering © 2009, USC-CSSE 3 Objectives and Background

4. University of Southern California Center for Systems and Software Engineering © 2009, USC-CSSE 4 Objectives Analysis of Productivity –How the productivity of the COCOMO data projects has changed over the years? –What caused the changes in productivity? Analysis of COCOMO cost drivers –How cost driver ratings have changed over the periods? –Are there any implications from these changes? Inviting further discussions

5. University of Southern California Center for Systems and Software Engineering © 2009, USC-CSSE 5 Estimation models need upgrading It has been 10 years since the release of COCOMO II.2000 –Data collected during 1970 – 1999 Software engineering practices and technologies are changing –Process: CMM  CMMI, ICM, agile methods –Tools are more sophisticated –Advanced communication facility Improved storage and processing capability

6. University of Southern California Center for Systems and Software Engineering © 2009, USC-CSSE 6 COCOMO Data Projects Over the Five-year Periods Dataset has 255 projects completed between 1970 and 2009 –161 used in COCOMO II 2000 –87 collected since 2000 7 projects were collected before 2000 but not included in COCOMO II 2000

7. University of Southern California Center for Systems and Software Engineering © 2009, USC-CSSE 7 COCOMO II Formula Effort estimate (PM) –COCOMO II 2000: A and B constants were calibrated using 161 data points with A = 2.94 and B = 0.91 Productivity = Constant A is considered as the inverse of adjusted productivity

8. University of Southern California Center for Systems and Software Engineering © 2009, USC-CSSE 8 Changes in Productivity and Cost Drivers

9. University of Southern California Center for Systems and Software Engineering © 2009, USC-CSSE 9 Average productivity is increasing over the periods Two productivity increasing trends exist: 1970 – 1994 and 1995 – 2009 1970-1974 1975-1979 1980-1984 1985-1989 1990-1994 1995-1999 2000-2004 2005-2009 Five-year Periods SLOC per PM 1970-1999 productivity trends largely explained by cost drivers and scale factors Post-2000 productivity trends not explained by cost drivers and scale factors

10. University of Southern California Center for Systems and Software Engineering © 2009, USC-CSSE 10 Effort Multipliers and Scale Factors EM’s and SF’s don’t change sharply as does the productivity over the periods EAF 1970- 1975- 1980- 1985- 1990- 1995- 2000- 2005- 1974 1979 1984 1989 1994 1999 2004 2009 Sum of Scale Factors 1970- 1975- 1980- 1985- 1990- 1995- 2000- 2005- 1974 1979 1984 1989 1994 1999 2004 2009 Effort Adjustment Factor (EAF) or ∏EM Sum of Scale Factors (  SF)

11. University of Southern California Center for Systems and Software Engineering © 2009, USC-CSSE 11 Changes in Product Factors and Platform Factors Testing data (DATA) increases over the years, suggesting an increase in size of the testing data Storage constraint (STOR) is almost constant in the last three periods Storage constraint was important during 1970 – 1979, but not during the other periods 1970- 1975- 1980- 1985- 1990- 1995- 2000- 2005- 1974 1979 1984 1989 1994 1999 2004 2009 1970- 1975- 1980- 1985- 1990- 1995- 2000- 2005- 1974 1979 1984 1989 1994 1999 2004 2009

12. University of Southern California Center for Systems and Software Engineering © 2009, USC-CSSE 12 Changes in Product Factors and Platform Factors Programmer capability is low in the 70-74 period Language and platform experience seems improving Application experience seems declining –Programmers change projects more often? Tool support gradually increases over the years –20% improvement in effort over 40 years Multisite development drops sharply in the last period –Supported by technologies? 1970- 1975- 1980- 1985- 1990- 1995- 2000- 2005- 1974 1979 1984 1989 1994 1999 2004 2009 1970- 1975- 1980- 1985- 1990- 1995- 2000- 2005- 1974 1979 1984 1989 1994 1999 2004 2009 Personnel Factors

13. University of Southern California Center for Systems and Software Engineering © 2009, USC-CSSE 13 Constant A decreases over the periods Calibrate the constant A while stationing B = 0.91 Constant A is the inverse of adjusted productivity –adjusts the productivity with SF’s and EM’s Constant A decreases over the periods 1970- 1975- 1980- 1985- 1990- 1995- 2000- 2005- 1974 1979 1984 1989 1994 1999 2004 2009 Productivity is not fully characterized by SF’s and EM’s 50% decrease over the post- 2000 period What factors can explain the phenomenon?

14. University of Southern California Center for Systems and Software Engineering © 2009, USC-CSSE 14 Analysis of Productivity Ranges Productivity range is the ratio between the highest and lowest rating value for a cost driver –Represents the degree of impact of the cost driver on effort Calibrated SF’s and EM’s by dividing the dataset into five sub-datasets –161 projects used in COCOMO II 2000 completed and collected by 1999 –190 projects completed between 1990 – 2009 –51 maintenance projects completed between 1989 – 2009 –149 new development projects completed between 1989 – 2009

15. University of Southern California Center for Systems and Software Engineering © 2009, USC-CSSE 15 PR of Cost Drivers of COCOMO II and all projects in 1990 - 2009 Major changes: - Increase  CPLX - Decrease Θ DATA, ACAP, PMAT 1990 – 2009: including 97 projects used in COCOMO II.2000 COCOMO II.2000 1. Complexity is more influential in the more recent data 2. Analyst capability and process are less influential in the more recent data 3. Testing data is least influential 1. Complexity is more influential in the more recent data 2. Analyst capability and process are less influential in the more recent data 3. Testing data is least influential

16. University of Southern California Center for Systems and Software Engineering © 2009, USC-CSSE 16 PR of Cost Drivers of New Development and Maintenance Major changes: - Increase  DATA, DOCU - Decrease Θ PCON, ACAP, PCAP, CPLX, PREC New Development Maintenance 1. Testing data and documentation are more important in maintenance 2. Personnel capacity, complexity in maintenance is not as critical as in development 1. Testing data and documentation are more important in maintenance 2. Personnel capacity, complexity in maintenance is not as critical as in development

17. University of Southern California Center for Systems and Software Engineering © 2009, USC-CSSE 17 Discussions Productivity has doubled over the last 40 years –But scale factors and effort multipliers did not fully characterize this increase Hypotheses/questions for explanation –Is standard for rating personnel factors different among the organizations? –Were automatically translated code reported as new code? –Were reused code reported as new code? –Are the ranges of some cost drivers not large enough? Improvement in tools (TOOL) only contributes to 20% reduction in effort –Are more lightweight projects being reported? Documentation relative to life-cycle needs

18. University of Southern California Center for Systems and Software Engineering © 2009, USC-CSSE 18 Conclusions Productivity is generally increasing over the 40- year period –SF’s and EM’s only partially explain this improvement The impact of some cost drivers on effort has changed when calibrating more recent data –Increased impact of complexity (CPLX) –Decreased impact of testing data (DATA), Analyst capability (ACAP), and Process capability (PMAT) Need for further analysis and calibration using recent data points