University of Southern California Center for Systems and Software Engineering © 2009, USC-CSSE 1 An Analysis of Changes in Productivity and COCOMO Cost Drivers over the Years Vu Nguyen, Barry Boehm Center for Systems and Software Engineering (CSSE) 24 th Int’l COCOMO Forum Nov 3 rd, 2009

University of Southern California Center for Systems and Software Engineering © 2009, USC-CSSE 2 Outline Objectives and Background Productivity Changes Discussions and Conclusions Cost Driver Changes

University of Southern California Center for Systems and Software Engineering © 2009, USC-CSSE 3 Objectives and Background

University of Southern California Center for Systems and Software Engineering © 2009, USC-CSSE 4 Objectives Analysis of Productivity –How the productivity of the COCOMO data projects has changed over the years? –What caused the changes in productivity? Analysis of COCOMO cost drivers –How cost driver ratings have changed over the periods? –Are there any implications from these changes? Inviting further discussions

University of Southern California Center for Systems and Software Engineering © 2009, USC-CSSE 5 Estimation models need upgrading It has been 10 years since the release of COCOMO II.2000 –Data collected during 1970 – 1999 Software engineering practices and technologies are changing –Process: CMM  CMMI, ICM, agile methods –Tools are more sophisticated –Advanced communication facility Improved storage and processing capability

University of Southern California Center for Systems and Software Engineering © 2009, USC-CSSE 6 COCOMO Data Projects Over the Five-year Periods Dataset has 255 projects completed between 1970 and 2009 –161 used in COCOMO II 2000 –87 collected since projects were collected before 2000 but not included in COCOMO II 2000

University of Southern California Center for Systems and Software Engineering © 2009, USC-CSSE 7 COCOMO II Formula Effort estimate (PM) –COCOMO II 2000: A and B constants were calibrated using 161 data points with A = 2.94 and B = 0.91 Productivity = Constant A is considered as the inverse of adjusted productivity

University of Southern California Center for Systems and Software Engineering © 2009, USC-CSSE 8 Changes in Productivity and Cost Drivers

University of Southern California Center for Systems and Software Engineering © 2009, USC-CSSE 9 Average productivity is increasing over the periods Two productivity increasing trends exist: 1970 – 1994 and 1995 – Five-year Periods SLOC per PM productivity trends largely explained by cost drivers and scale factors Post-2000 productivity trends not explained by cost drivers and scale factors

University of Southern California Center for Systems and Software Engineering © 2009, USC-CSSE 10 Effort Multipliers and Scale Factors EM’s and SF’s don’t change sharply as does the productivity over the periods EAF Sum of Scale Factors Effort Adjustment Factor (EAF) or ∏EM Sum of Scale Factors (  SF)

University of Southern California Center for Systems and Software Engineering © 2009, USC-CSSE 11 Changes in Product Factors and Platform Factors Testing data (DATA) increases over the years, suggesting an increase in size of the testing data Storage constraint (STOR) is almost constant in the last three periods Storage constraint was important during 1970 – 1979, but not during the other periods

University of Southern California Center for Systems and Software Engineering © 2009, USC-CSSE 12 Changes in Product Factors and Platform Factors Programmer capability is low in the period Language and platform experience seems improving Application experience seems declining –Programmers change projects more often? Tool support gradually increases over the years –20% improvement in effort over 40 years Multisite development drops sharply in the last period –Supported by technologies? Personnel Factors

University of Southern California Center for Systems and Software Engineering © 2009, USC-CSSE 13 Constant A decreases over the periods Calibrate the constant A while stationing B = 0.91 Constant A is the inverse of adjusted productivity –adjusts the productivity with SF’s and EM’s Constant A decreases over the periods Productivity is not fully characterized by SF’s and EM’s 50% decrease over the post period What factors can explain the phenomenon?

University of Southern California Center for Systems and Software Engineering © 2009, USC-CSSE 14 Analysis of Productivity Ranges Productivity range is the ratio between the highest and lowest rating value for a cost driver –Represents the degree of impact of the cost driver on effort Calibrated SF’s and EM’s by dividing the dataset into five sub-datasets –161 projects used in COCOMO II 2000 completed and collected by 1999 –190 projects completed between 1990 – 2009 –51 maintenance projects completed between 1989 – 2009 –149 new development projects completed between 1989 – 2009

University of Southern California Center for Systems and Software Engineering © 2009, USC-CSSE 15 PR of Cost Drivers of COCOMO II and all projects in Major changes: - Increase  CPLX - Decrease Θ DATA, ACAP, PMAT 1990 – 2009: including 97 projects used in COCOMO II.2000 COCOMO II Complexity is more influential in the more recent data 2. Analyst capability and process are less influential in the more recent data 3. Testing data is least influential 1. Complexity is more influential in the more recent data 2. Analyst capability and process are less influential in the more recent data 3. Testing data is least influential

University of Southern California Center for Systems and Software Engineering © 2009, USC-CSSE 16 PR of Cost Drivers of New Development and Maintenance Major changes: - Increase  DATA, DOCU - Decrease Θ PCON, ACAP, PCAP, CPLX, PREC New Development Maintenance 1. Testing data and documentation are more important in maintenance 2. Personnel capacity, complexity in maintenance is not as critical as in development 1. Testing data and documentation are more important in maintenance 2. Personnel capacity, complexity in maintenance is not as critical as in development

University of Southern California Center for Systems and Software Engineering © 2009, USC-CSSE 17 Discussions Productivity has doubled over the last 40 years –But scale factors and effort multipliers did not fully characterize this increase Hypotheses/questions for explanation –Is standard for rating personnel factors different among the organizations? –Were automatically translated code reported as new code? –Were reused code reported as new code? –Are the ranges of some cost drivers not large enough? Improvement in tools (TOOL) only contributes to 20% reduction in effort –Are more lightweight projects being reported? Documentation relative to life-cycle needs

University of Southern California Center for Systems and Software Engineering © 2009, USC-CSSE 18 Conclusions Productivity is generally increasing over the 40- year period –SF’s and EM’s only partially explain this improvement The impact of some cost drivers on effort has changed when calibrating more recent data –Increased impact of complexity (CPLX) –Decreased impact of testing data (DATA), Analyst capability (ACAP), and Process capability (PMAT) Need for further analysis and calibration using recent data points